No internet connection
  1. Home
  2. Support

Webhooks – last piece of the puzzle for full service integration

By Christian Scheuer @chrscheuer
    2021-10-04 21:46:22.316Z

    Hi @KajMagnus,

    I just wanted to sync up with you to see if getting webhooks support for certain events would be possible in the (near) future?

    Right now we can only download the entire json structure of the forum to allow support posts made in the forum to sync with our other services (ie. Linear, Intercom, Firebase, etc.)

    But because we have to download the entire json structure, this is very slow and can only be scheduled a few times per day at the max - otherwise I feel like we'd risk overloading your server by asking it to fetch everythingall the time.
    Beyond that, because of this delay syncing can't be made really effective - as our other systems would always be out of sync with the real data in the forum, because it would easily be up to 24 hours late in arriving.

    Because of this, we are currently not integrating the forum as deeply to our services as we'd like.

    Since your last great changes, we can now create posts in the forum impersonating a user. That's great :) So the last piece of the puzzle is the ability to react to things happening in Talkyard in real-time via webhooks.

    For info, Stripe has an excellent way of handling webhooks.

    All well run webhook integrations I've seen do it like this:

    • Have a place in the admin UI to add webhook endpoints. You typically just specify a URL, and sometimes you can opt-in to the types of events you want the webhook to receive. This is useful if the system can send many different event types and some webhooks may want to just subscribe to a few event types.
    • Record an event object in the database whenever something insteresting occurs.
    • Try sending this event to all registered webhooks matching that event and record if the events were processed correctly (webhook responded 200 OK within a timeout period)
    • If any of the webhooks failed for the event, retry sending it at a later time, typically with exponential rolloff.
    • Present a way to report in an admin UI which events were sent successfully and which had errors etc. Perhaps even the ability to show details about each event.
    • Event objects/payloads are typically sent via JSON, and often the event would contain info like:
      • Event type
      • Time
      • Idempotency Key (see Stripe docs for this – if for example the event is the result of an API call where you specified an idempotency key, this would be returned in the event created by the result of that API call, to let the external site know that the effect of the async API call was now received). Optional
      • The entity the event is about. For example, the customer, or the post, or the page, etc.
      • Other potential metadata

    The event types we'd be interested in would be:

    • Page gets created or updated
    • Post gets created or updated
    • Status of a page changes (closed, reopened)

    Obviously such a webhook implementation can be made quite complex. But I wonder if we could get something off the ground if we skipped some of these more thorough implementation details for now and opted for a simpler approach to get started.

    One that just as a bare minimum:

    • Sends a single HTTPS request to the webhook endpoint when one of the above mentioned events occurs.
    • It could be done with a simple, no retrying, no event persistence, no event admin UI implementation as a first step.

    Would you consider implementing something like this to make TY able to integrate into these other types of systems?

    • 7 replies
    1. C
      Christian Scheuer @chrscheuer
        2021-10-04 21:50:06.498Z

        Here's an example of how Stripe displays recorded events.

        And for each event it will show us details about the contents of the event as well as the success/failure of attempts of sending that event to the webhooks registered in the account.

        1. In reply tochrscheuer:

          I wonder if we could get something off the ground if we skipped some of these more thorough implementation details for now and opted for a simpler approach to get started.

          Yes: I'm thinking about implementing an /-/v0/events endpoint first. It's basically only the "Record an event object in the database" step (or not even that), plus an API endpoint. And, from what I've understood, it's needed in any case, as a complement to webhooks.

          Stripe has such an endpoint: https://stripe.com/docs/api/events

          A discussion at HackerNews: "Give me /events, not webhooks" https://news.ycombinator.com/item?id=27823109.
          Quoting the blog post: "In general, you can't rely on webhooks alone to keep two systems consistent".
          The first reply in the HackerNews discussion is from someone at Stripe.

          /-/v0/events is via polling. You could poll as often as you want (e.g. every second); the load would be negligible (proportional to number of Ty sites, rather than Ty sites * members). Later, we could add webhooks as well.

          (Thanks for the writeup about webhooks :- )   A good list of things to think about. — There's also versioning of the webhook payload format, maybe that could be a per site and webhook setting. It's mentioned in the HackerNews discussion b.t.w.)

          1. CChristian Scheuer @chrscheuer
              2021-10-05 13:27:52.873Z

              Got it, yea if the endpoint supports:

              • Give me the events that happened since this last event that I have the ID of.

              Then that would work great. We'd then make a note of the last event we processed and always just ask for events newer than that.
              There's some complexity around corner cases which we'd end up having to implement on our end, so ideally, when you then do webhooks later, it would limit our exposure to that complexity and deal with it for us.

              1. CChristian Scheuer @chrscheuer
                  2021-10-05 13:28:24.031Z

                  Wrt versioning of webhook formats. Absolutely, should be controllable per site and/or per webhook. That's how Stripe does it.

                  1. CChristian Scheuer @chrscheuer
                      2021-10-05 13:30:08.208Z

                      I gotcha on /events and them being needed for consistency, but IMO webhooks are simpler to implement for those of us who don't want to mess with cron jobs if we don't have to :)

                      Since we're on GCP we already have some cronjobs running though, so it would be ok. But implementing polling will make the solution potentially quite a bit more expensive for us to run as a starting point.
                      The reason being that we use Cloud Run which bills per CPU milliseconds spent, so if the webhook would only send us a message, say, every 20 seconds, we'd be billed for those few milliseconds it takes to process the message, once every 20 seconds. With a real-time webhook implementation we'd need to run a cron job for example every 2 seconds (if we want real-time behavior). This would give us approx 10x the bill.

                      We can live with that though :) Just so you know why others might want to favor webhooks.

                      1. CChristian Scheuer @chrscheuer
                          2021-10-05 13:38:44.349Z

                          Wooop!! You tagged this! How are tags coming along? Soo excited for this :)

                  2. In reply tochrscheuer:

                    a simple, no retrying, no event persistence, no event admin UI implementation as a first step.

                    That sounds like a good starting point for webhooks. Hmm, some persistence would actually be needed, since Ty .net is a SaaS — a pranks minded person could decide to send webhooks to the wrong server, causing unexpected weird things to happen. And then the Ty server would need to remember for example that "oops, that server doesn't want my webhooks".