[Feature Request] Accept Jaeger as an input

[worldtiki]

This was asked in the Haystack gitter room.

I'm not entirely sure how this would work at this point, ie, do we want http ingress, or kafka/etc as well but the goal would be to adapt Pitchfork to also support tracing data in the format used by Jaeger.

[codefromthecrypt]

suppose this was to help migrate off jaeger to haystack?

[worldtiki]

I don't know how to paste a link to a gitter room but this is related to a reply from @mchandramouli on May 02 to @dannashirn in the #haystack/Lobby room.

One option would be to add a collector in Haystack to ingest Jaeger spans, the other would be to add one to Pitchfork.

I think it would make sense if Pitchfork supported this, however I have some reservations about how easy this would be to maintain if formats start to diverge, not just between Jaeger and Haystack but between Jaeger and Zipkin too as the whole Pitchfork code is based on zipkin2.Span objects.
Right now there's has a 1 to many input/output options where we accept Zipkin and output to Zipkin+Haystack.
By supporting Jaeger we would have a many to many where we'd accept Zipkin+Jaeger and output to Zipkin+Haystack.

I think I'll need to play with this for a while and explore some different options and I guess we could drop the output to Zipkin since I believe that was very specific to our use case at Hotels.com.

[codefromthecrypt]

depends on what you mean by "I guess we could drop the output to Zipkin" If I were you, I would check the issues list in case folks have asked about it and if was only internal, rename the project to be a rosetta stone for haystack ingest.

It is super odd that instrumentation cannot just send in haystack format, I mean wasn't that the wholepoint of opentracing?

[mchandramouli]

@adriancole @worldtiki - They have existing instrumentation that is sending spans in OT / Jeagar in JSON format. We wanted to convert to proto. Biggest problem with OT is not having a datamodel / transport format. We could have done it as json -> proto Haystack's collector or do it Pitchfork to allow incoming OT json to be used in Haystack+Zipkin combined stack. Since Pitchfork does json to proto I thought it will be an easy addition. Pitchfork is key to have Zipkin+Haystack working together - wouldnt recommend changing that.

Not sure if this is valid request anymore as after I posted the request to @worldtiki - I see @dannashirn has created a simple json to proto kstreams transformer. I guess that unblocks them.