No internet connection
  1. Home
  2. Support

How to migrate from the Talkyard-hosted server to a self-hosted one?

By Christian Scheuer @chrscheuer
    2022-04-14 18:55:00.698Z

    We need a way to be in more control over the uptime of the forum. As an example, search has been broken for several hours now, and other systems are affected too, and there doesn't appear to be any way to get a hold of @KajMagnus.

    Please note, we're happy to keep supporting the platform financially, this is not about cost, but about being in control of uptime.

    I have set up a self hosted Ubuntu server now - by the way congrats on making those steps so easy to follow! Great installation experience.

    How would we go about migrating the Talkyard-hosted server to our own server? I know this is likely a semi big operation, but this would be important to us to get done to ensure stability.

    Solved in post #9, click to view
    • 10 replies
    1. C
      Christian Scheuer @chrscheuer
        2022-04-14 20:07:49.398Z

        I'm trying to build the "to-talkyard" node.js, but it seems broken. When running "yarn build" in the "to-talkyard" directory I'm getting all sorts of missing dependencies.
        It feels like this hasn't been properly tested on a fresh git repo / fresh installation.

        1. CChristian Scheuer @chrscheuer
            2022-04-14 20:20:54.985Z

            Got it building. You need to run "yarn" in the main talkyard directory first.

            What I'm not entirely sure about is if the "site-export-json" format is supported by the "to-talkyard" script for importing the data.

            Otherwise, I'd appreciate if @KajMagnus could help with the transfer of our data to a self-hosted forum perhaps by making a direct export from the database.

            1. CChristian Scheuer @chrscheuer
                2022-04-14 20:26:57.696Z

                I just noticed there's an /-/import-site-json endpoint. Unfortunately, our exported JSON is approx. 150 MB in size, and it appears we can't send so large payloads to the import-site-json endpoint.

                1. CChristian Scheuer @chrscheuer
                    2022-04-14 20:32:39.859Z

                    Even when modifying the environment variable in the docker-compose.yml to accept large payloads, I'm still receiving

                     <h1>Bad Request</h1>
                    
                            <p id="detail">
                                For request 'POST /-/import-site-json?deleteOldSite=true' [Request Entity Too Large]
                            </p>
                    
                    1. CChristian Scheuer @chrscheuer
                        2022-04-14 21:35:18.004Z

                        Hm, I may have managed to bump enough limits to make this work, but it's been running for over an hour now so not sure if it's still just busy importing or if it's failing/failed.

                        1. CChristian Scheuer @chrscheuer
                            2022-04-14 21:37:46.078Z

                            Hm it broke down due to lack of memory (I had 8 GB RAM allocated). @KajMagnus I'm hoping you can help do a more raw site import/export with the direct database contents to make this process easier.

                            1. CChristian Scheuer @chrscheuer
                                2022-04-14 21:41:58.100Z

                                A related question: What's the current resources the TY production server uses (vCPU, RAM, disk type/size)? Just so we can make sure to at least provide the same config.
                                KajMagnus in terms of still supporting TY financially (which we want to do, we want to support your great work here) yet hosting ourselves, please be in touch over email (I sent you an email about the urgent problem earlier).

                    2. In reply tochrscheuer:

                      I'll help you with this, I'll write more a bit later, probably in a PM, and then even later we / I can post an update here at Ty .io

                      Reply1 LikeSolution
                      1. CChristian Scheuer @chrscheuer
                          2022-04-18 17:10:09.118Z2022-04-18 17:33:29.491Z

                          Thank you so much for all the help!
                          To anybody else reading this in the future - Magnus helped us doing the full migration and spent hours and hours over a weekend to get everything up and running. Outstanding level of support!

                          1. Thanks :- )   (Now, with some scripts and procedures written for this, the next time someone wants to migrate to self hosted, that'll be simpler. And eventually, some day in the distant future, automatic so one just clicks a button and gets a database & uploads dump, and a how-to readme, in an encrypted .tar.gz)