A template for self-hosting Grist with traefik and docker compose

Yohan_Boniface · September 14, 2022, 10:42am

@paul-grist two more questions:

do you have a recommendation for the RAM on the server ? We hit 1Go yesterday, so we now have passed to 2Go, FYI.
do you think it’s possible to have multiple containers running in parallel (i.e. is a running Grist stateless) ? Would that need using PostgreSQL for the “core” db ?

Thanks for your lights!

Yohan_Boniface · September 14, 2022, 3:18pm

For the record, we just hit the 2 Go RAM.

paul-grist · September 14, 2022, 3:33pm

Hi Yohan, there are some hardware recommendations at Self-managed Grist - Grist Help Center

It is indeed possible to have multiple containers in parallel - we do so in our hosted service - but there’s significant engineering work needed to make this happen. In general for a single organization I’d expect one server with more CPU and RAM to be preferable to multiple lighter servers.

dmitry-grist · November 21, 2023, 10:14pm

Security Update 2023-11-21

The original version of the instructions above, combined with a version of grist-core up until v1.1.7, produced an insecure configuration which made it possible for an attacker to impersonate any user on the system.

Please be sure to update grist-core and/or the gristlabs/grist docker image to the latest fixed version (v1.1.8, stable, or main).

This alert also applies to grist-omnibus (docker image gristlabs/grist-omnibus). Upgrading to the latest version will fix the issue.

The issue affected specifically the suggested configuration of grist-core with traefik. If you used a different configuration, you may not be affected.

yala1 · September 28, 2024, 6:24pm

Based on this example with Traefik, we are regularly using and updating Grist with this configuration:

compose.yml

networks:
  internal: {}
  web:
    external: true

services:

  grist:
    image: gristlabs/grist
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      - GRIST_DOMAIN
      - APP_HOME_URL

      - REDIS_URL

      - TYPEORM_TYPE
      - TYPEORM_HOST
      - TYPEORM_DATABASE
      - TYPEORM_USERNAME
      - TYPEORM_PASSWORD

#      - GRIST_DOCS_MINIO_ENDPOINT
#      - GRIST_DOCS_MINIO_USE_SSL
#      - GRIST_DOCS_MINIO_BUCKET_REGION
#      - GRIST_DOCS_MINIO_BUCKET
#      - GRIST_DOCS_MINIO_ACCESS_KEY
#      - GRIST_DOCS_MINIO_SECRET_KEY

      - GRIST_OIDC_SP_HOST
      - GRIST_OIDC_IDP_ISSUER
      - GRIST_OIDC_IDP_CLIENT_ID
      - GRIST_OIDC_IDP_CLIENT_SECRET
      - GRIST_OIDC_IDP_SKIP_END_SESSION_ENDPOINT
      - GRIST_ANON_PLAYGROUND

      - GRIST_DEFAULT_EMAIL
      - GRIST_PAGE_TITLE_SUFFIX
      - GRIST_SESSION_SECRET
      - GRIST_HOME_INCLUDE_STATIC
      - GRIST_ORG_IN_PATH

      - GRIST_SUPPORT_EMAIL
      - GRIST_HIDE_UI_ELEMENTS
      - GRIST_SANDBOX_FLAVOR
      - GRIST_EXPERIMENTAL_PLUGINS
      - GRIST_BOOT_KEY
      - GRIST_WIDGET_LIST_URL

      - ALLOWED_WEBHOOK_DOMAINS
      - COMMENTS
    volumes:
      - "./.state/grist:/persist"
    networks:
      - internal
      - web
    healthcheck:
      test: ["CMD-SHELL", "python -c \"import urllib.request; req=urllib.request.Request('http://127.0.0.1:8484/api/orgs'); req.add_header('Host', '${GRIST_DOMAIN}'); urllib.request.urlopen(req).getcode()\""]
      start_period: 10s
      interval: 10s
      retries: 5
      timeout: 3s
    labels:
      traefik.enable: true
      traefik.http.middlewares.permanent-redirect.redirectscheme.scheme:    https
      traefik.http.middlewares.permanent-redirect.redirectscheme.permanent: true
      traefik.http.routers.org-example-grist-redirect.entrypoints:           web
      traefik.http.routers.org-example-grist-redirect.middlewares:           permanent-redirect
      traefik.http.routers.org-example-grist-redirect.rule:                  Host(`${FQDN}`)

      traefik.http.services.org-example-grist.loadbalancer.server.port: 8484

      traefik.http.routers.org-example-grist-general.rule:             Host(`${FQDN}`)
      traefik.http.routers.org-example-grist-general.entrypoints:      web-secure
      traefik.http.routers.org-example-grist-general.service:          org-example-grist
      traefik.http.routers.org-example-grist-general.tls:              true
      traefik.http.routers.org-example-grist-general.tls.certresolver: letsencrypt

  postgres:
    image: postgres:15-alpine
    command: -c jit=off
    environment:
      - POSTGRES_USER
      - POSTGRES_PASSWORD
      - POSTGRES_DB
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
      - ./.state/postgres:/var/lib/postgresql/data
    networks:
      - internal
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "${POSTGRES_USER}"]
      start_period: 10s
      interval: 10s
      retries: 5
      timeout: 3s

  redis:
    image: redis:6-alpine
    command: ["redis-server","/usr/local/etc/redis/redis.conf"]
    environment:
      - REDIS_URL
    volumes:
      - "./.state/redis:/data"
      - "./redis.conf:/usr/local/etc/redis/redis.conf"
    networks:
      - internal
    healthcheck:
      test: ["CMD-SHELL", "redis-cli -u ${REDIS_URL} ping | grep 'PONG' || exit 1"]
      start_period: 10s
      interval: 10s
      retries: 5
      timeout: 3s

We’re not (yet) using Minio with a snapshotting backend, why it is commented out.

redis.conf

cat > redis.conf <<< "requirepass $(openssl rand -base64 48)"

.env

FQDN=grist.example.org
EMAIL=anonymous@example.org

GRIST_DOMAIN=${FQDN}
APP_HOME_URL=https://${FQDN}

COOKIE_SECRET=<openssl rand -base64 48>
GRIST_SESSION_SECRET=<openssl rand -base64 48>

GRIST_BOOT_KEY=<openssl rand -base64 48>

POSTGRES_DB=org-example-grist
POSTGRES_USER=org-example-grist
POSTGRES_PASS=<openssl rand -base64 48>

TYPEORM_TYPE=postgres
TYPEORM_HOST=postgres
TYPEORM_DATABASE=${POSTGRES_DB}
TYPEORM_USERNAME=${POSTGRES_USER}
TYPEORM_PASSWORD=${POSTGRES_PASS}

REDIS_SECRET=<secret from redis.conf>
REDIS_URL=redis://default:${REDIS_SECRET}@redis:6379/1

# Requires MinIO w/ EC, at least 4 XFS drives
# GRIST_DOCS_MINIO_ENDPOINT=minio.example.org
# GRIST_DOCS_MINIO_USE_SSL=true
# GRIST_DOCS_MINIO_BUCKET_REGION=eu-central-1
# GRIST_DOCS_MINIO_BUCKET=org-example-grist
# GRIST_DOCS_MINIO_ACCESS_KEY=org.example.grist
# GRIST_DOCS_MINIO_SECRET_KEY=<openssl rand -base64 48>

GRIST_OIDC_SP_HOST=https://${FQDN}
GRIST_OIDC_IDP_ISSUER=https://gitlab.example.org/.well-known/openid-configuration
GRIST_OIDC_IDP_CLIENT_ID=
GRIST_OIDC_IDP_CLIENT_SECRET=
GRIST_OIDC_IDP_SKIP_END_SESSION_ENDPOINT=true
GRIST_ANON_PLAYGROUND=false

GRIST_DEFAULT_EMAIL=${EMAIL}
GRIST_PAGE_TITLE_SUFFIX=_blank
GRIST_HOME_INCLUDE_STATIC=true
GRIST_ORG_IN_PATH=true

GRIST_SUPPORT_EMAIL=support@example.org
GRIST_HIDE_UI_ELEMENTS=helpCenter,billing
#GRIST_HIDE_UI_ELEMENTS=helpCenter,billing,templates,multiSite,multiAccounts
GRIST_SANDBOX_FLAVOR=unsandboxed
#GRIST_SANDBOX_FLAVOR=gvisor
GRIST_EXPERIMENTAL_PLUGINS=true
GRIST_WIDGET_LIST_URL=https://github.com/gristlabs/grist-widget/releases/download/latest/manifest.json

ALLOWED_WEBHOOK_DOMAINS=n8n.example.org,mattermost.example.org,gitlab.example.org
COMMENTS=true

This should get everyone started with a Grist instance behind Traefik. Ping me, if you have any questions or comments.

I’m posting this now, because I have just come up with this healthcheck: for the Grist container. This or something similar could also be added to the Dockerfile as a HEALTHCHECK line, in so all container schedulers can benefit from it’s presence. The easiest would probably to also add a minor curl dependency on the final container image and check the status code of the /api/orgs route (the only one without parameters) with it instead of this Python one-liner construct.

paul-grist · October 1, 2024, 2:55pm

Thanks for this @yala1 !

I see GRIST_SANDBOX_FLAVOR=unsandboxed in there which is reasonable if you trust all documents (glad to see GRIST_ANON_PLAYGROUND=false) otherwise it could be dangerous. The options gvisor or pyodide offer some sandboxing of user Python code.

There is also an undocumented /status endpoint for the purpose of healthchecks.

github.com

gristlabs/grist-core/blob/77194dcb200531bb5a98cf052444aaca9d01bfe4/app/server/lib/FlexServer.ts#L493-L502


      
          public addHealthCheck() {
            if (this._check('health')) { return; }
            // Health check endpoint. if called with /hooks, testing hooks are required in order to be
            // considered healthy.  Testing hooks are used only in server started for tests, and
            // /status/hooks allows the tests to wait for them to be ready.
            // If db=1 query parameter is included, status will include the status of DB connection.
            // If redis=1 query parameter is included, status will include the status of the Redis connection.
            // If docWorkerRegistered=1 query parameter is included, status will include the status of the
            // doc worker registration in Redis.
            this.app.get('/status(/hooks)?', async (req, res) => {

Adding curl into the image would be very reasonable - we would accept a pull request for that, otherwise I expect we’ll get to this eventually.

yala1 · October 2, 2024, 7:07am

Many thanks for the review and comment on security implications especially.

I’m having many tabs open with Grist at the moment, thanks to your casual reminders that community contributions are accepted. Will be happy to pick up some of the loose ends nearing winter.