No internet connection
  1. Home
  2. Ideas

Talkyard List API

By KajMagnus @KajMagnus2020-05-21 03:21:10.478Z2020-05-21 03:30:15.726Z

There's a search API: POST /-/v0/search for full text search, and also an API at: POST /-/v0/list for listing for example the X most popular pages in a category, or the Y most recent posts across the whole forum. This topic is about the List API — what things do people want to list, and what data should be included in the reply.

Here's an API request example, for listing the X most recently created pages in a category:

POST /-/v0/list  {
  listQuery: {
    findWhat: 'Pages',
    inCategories: ['extid:your_category_id'],
    sortOrder: 'NewestFirst',
    limit: number,
  }
}
  • 7 replies
  1. KajMagnus @KajMagnus2020-05-21 21:52:03.417Z2020-05-21 22:00:52.203Z

    Andreas @pieber, you wrote: (here)

    When a new topic is created a new ticket should be created in my tracker
    When a comment is added to a topic a comment should be posted to an existing ticket.

    What information about new topics and replies would you want, in order to do that?
    For example, what about topic category id?
    Tags? (later, when re-implemented)
    Topic type? (question / problem / idea / discussion / chat)
    Author info — name? Author id somehow?
    Topic title and text I suppose? And reply texts. Reply number and parent number? (so knows who replies to who)

    Let's say two topics in Talkyard have the exact same title — then, how will the bug tracker know they're two different topics?
    Talkyard can include a pageId field, so you could use that. (Does the tracker have some way to associate tickets with external ids?)

    I'm curious about which bug tracker you use? (it's a bug tracker?)

    1. PAndreas Pieber @pieber
        2020-05-22 09:45:47.750Z

        I'm curious about which bug tracker you use? (it's a bug tracker?)

        We're using YouTrack from Jetbrains.

        Talkyard can include a pageId field, so you could use that. (Does the tracker have some way to associate tickets with external ids?)

        Actually, you can create any field. I would have planned to attach a page ID to the ticket. That way I could query if there is an issue already or if I need to create one.

        What information about new topics and replies would you want, in order to do that?

        The bare minimum I need to a new topic:

        • Author (ID, E-Mail and Full Name)
        • Creation Date Time
        • Topic Type
        • Topic Title
        • Topic text idallay already as rendered HTML (YouTrack supports Markdown and HTML - and since those can be entered in Talkyard mixed it helps to have a common format)
        • Category (name and id)

        I'm ok getting the same entry for any updated/changed information. For me it's not required if they're updated or new. I'll simply query if an entry exist and update or create it accordingly.

        BTW, the inCategories listQuery parameter isn't important for me since I always want to sync anything.

        Tags? (later, when re-implemented)

        I don't really care about tags since sorting, prioritizing and so on is something I do in the issue tracker.

        And reply texts. Reply number and parent number? (so knows who replies to who)

        For a reply the site-ID, a replyID/Number is essential. The parent-number would be nice to have since I don't have those deep reply structures in YouTrack as they're available in Talkyard. Author ID, Mail and Full-Name and the content as HTML again.

        I just though about something else. Since the sync service can always go down I need to re-sync an entire page. If I discover something missing I need to trigger a resync (e.g. if I get a response for a page I don't have stored, or a reply with a parent ID I don't have). In this case I need to get all of the information at once for a specific page (author, creation date, topic type, title, all responses and so on).

        I think with those features I should be able to implement the sync.

        1. Then there's a different endpoint I think is better for your use case:

          /-/export-site-json
          

          which gives you all you need. It's meant for exporting one's Talkyard site and importing to another Talkyard server, or to something else.

          text idallay already as rendered HTML

          There's a approvedHtmlSanitized field.

          I need to re-sync an entire page [...] get all of the information at once for a specific page

          This endpoint currently exports the whole site, so, currently, it gives you a bit too much instead.

          Maybe in the beginning, this doesn't matter much, since the Talkyard site wouldn't have that much contents, intially?

          Later there can be query param like: ?newerThan=2020-05-25T00:00:00 so you wouldn't get everything.
          Or ?pageId=... and then you'd get that page, the replies and authors.

          The json this endpoint returns looks like:

          {
            ...
            pages: [{
                "id": "8",   <—— unique id
                "pageType": 5,   <—— question / idea / problem / discussion
                "categoryId": 3,
                "createdAtMs": 1590397059126,
                "updatedAtMs": 1590397059126,
                "authorId": 100,
                "frequentPosterIds": [],
                ...
            }],
          
            categories: [{
                "id": 3,
                "sectionPageId": "1",
                "parentId": 1,
                "defaultSubCatId": null,
                "name": "Category Name",
                "slug": "category-name",
                "description": "Category description text.",
                "createdAtMs": 1590397023923,
                ...
            }],
          
            posts: [{   <—— replies, but also page titles and page bodies (orig posts)
                "id": 21,   <—— unique id
                "pageId": "8",
                "nr": 1,     <—— 0 = page title, 1 = Original Post. 2, 3, 4, ... = replies
                "parentNr": null,   <—— if replies to another post
                "createdAt": 1590397059126,
                "createdById": 100,   <—— author id
                "approvedSource": "Text text and a link: http://e2e-test--emb-exp-0397016.localhost:8080/impexp-subscr-notfs.html",
                "approvedHtmlSanitized": "<p>Text text and a link: <a href=\"http://e2e-test--emb-exp-0397016.localhost:8080/impexp-subscr-notfs.html\" rel=\"nofollow\">http://e2e-test--emb-exp-0397016.localhost:8080/impexp-subscr-notfs.html</a></p>\n",
                "approvedAt": 1590397059126,
                "approvedById": 1,
                "approvedRevNr": 1,
                ...
            }],
          
            members: [{
                "id": 100,
                "username": "owen_owner",
                "fullName": "Emb Cmts Exp test id 0397016",
                "emailAddress": "e2e-test--0397016@example.com",
                ...
            }],
            ..
          }
          

          To enable this endpoint go here: /-/admin/settings/features and check the Experimental checkbox, then go here: /-/admin/backup and click Download backup.

          This endpoint is not yet official. It hasn't changed the past year, so it's pretty stable — but some time it'll get renamed and the JSON format will change slightly.

          1. PAndreas Pieber @pieber
              2020-05-25 09:51:22.182Z

              This is perfect for now in case of an error (or if a sanity check fails) or initially. When this become a problem in the feature we can revisit this. I think I would still need an endpoint where I can get last pages/comments as discussed.

              1. In the upcoming version there's also /-/v0/list, so you can list the most recent pages and replies.

                The response currently doesn't include so detailed information though: No user emails or category names. Maybe I could add that (for email addresses: the requester would need to be admin).

                Currently the POST /-/v0/list { listQuery: { findWhat: 'Posts' }} response returns a list of PostListed:

                // (This is Typescript.)
                
                interface PostListed {
                  id: number;
                  nr: number;
                  parentNr?: number;
                
                  pageId: PageId;
                  pageTitle: string;
                
                  // maybe could include:
                  // categoryId,
                  // categoryName,
                
                  author: ParticipantFound;
                
                  urlPath: string;
                  approvedHtmlSanitized?: string;
                }
                
                interface ParticipantFound {
                  ppId: ParticipantId;
                  username?: string;
                  fullName?: string;
                  tinyAvatarUrl?: string;
                  isGroup?: boolean;
                  isGuest?: boolean;
                
                  // Maybe could include — only if the requester is admin / sysbot:
                  // emailAddress?: string,
                  // emailAddressVerified?: boolean,
                
                }
                
                1. PAndreas Pieber @pieber
                    2020-06-01 08:24:05.075Z

                    Category would be really nice - otherwise this will already work pretty well for my use case. If I upgrade to the latest master / WIP builds do they already include those features?

                    1. Category would be really nice

                      I start thinking this me too. Maybe usually, someone who lists the most recent posts, wants to show meta info about page title, author, and category.

                      If I upgrade to the latest master / WIP builds

                      Yes, actually your server should have auto upgraded itself yesterday (Sat - Sun night) to the latest version (which is v0.6.68-WIP-1-c255d72 == v0.6.68-c255d72). Currently without category.