Questions about self-hosted grist and authentik with OIDC

[Ovi]

I don’t even know where to start my questions
Let me preface this long thread by saying that I have read Self-managed Grist - Grist Help Center and most related pages and am still struggling to understand the whole concept related to teams, organizations, users and authentication. I’d appreciate any clarifications even simple links to documentation I might have missed.
if anyone is using authentik + OIDC I’d be very happy to compare the settings in authentik. I had to guess at most of them.

My first try at a self-hosted grist instance was using authentik as a reverse proxy authentication. This worked perfectly fine. I also added these envs to my compose file:

- GRIST_DEFAULT_EMAIL=me@domain.tld
- GRIST_SINGLE_ORG=sheets

This way, as soon as I logged in via authentik, I was logged in into grist, with this single email address and had access to the admin panel and the team called “sheets”

My next try was to use authentik as an OIDC provider and then I also experimented by enabling multiple team sites.

This meant removing these two envs:

- GRIST_DEFAULT_EMAIL=me@mydomain.tld
- GRIST_SINGLE_ORG=sheets

And adding these:

- GRIST_OIDC_IDP_ISSUER=https://authentik.mydomain.tld/application/o/grist/.well-known/openid-configuration
- GRIST_OIDC_IDP_CLIENT_ID=secretXXX
- GRIST_OIDC_IDP_CLIENT_SECRET=secretXXX
- GRIST_ORG_IN_PATH=true

This also worked in the sense that I could log into grist when logged into authentik. Since my authentik suer uses the same email as I used for GRIST_DEFAULT_EMAIL this worked just fine.

Now here are the confusing parts:

The login method does not reflect in my grist user profile:

I am now missing any link to the grist admin panel:

A new option to “add account” has appeared. To what/where do I add these accounts? To grist? To my team? This will not work anyway, since I guess it needs to be properly set up and redirected to some authentik signup page, which I haven’t done. Just asking for clarity’s sake.

Why does it say “legacy” next to the personal site? All I find about personal sites is this: Creating team sites - Grist Help Center - waht are they? Are they meant for sheets not shared with a team?

I just wanted to mention, that after signing out of both grist and authentik, and randomly clicking back and forth I somehow ended up on my grist instance as a guest. I didn’t even know this was possible, not sure how to get back here on purpose except for manually typing in the URL, but I am wondering what I can do as a guest? It looks like anyone can get here as a guest and add new docs?

[paul-grist]

@Ovi

GRIST_DEFAULT_EMAIL=me@domain.tld

@Ovi could you try setting this variable ^ again? Grist needs to know who the initial administrator is, and this is one way to set it.

For the question about guests, there’s a GRIST_ANON_PLAYGROUND that can control that functionality in multi-site operation. There’s also a GRIST_FORCE_LOGIN flag to require logins in all circumstances. See the README for details. I hope we’ll be surfacing and highlighting this behavior in the admin panel in the future, along with other security checks we’re adding now.

For the question about accounts. It is possible in a single browser session to log in as multiple accounts, and see all the sites you have access to. You need to be deep into Grist usage to care about this, and not likely to matter for you. From the README, I think you could hide the option by making GRIST_HIDE_UI_ELEMENTS contain multiAccounts, though I haven’t checked this.

Personal sites are a little space users have for documents that by default are private. The (Legacy) label worries me a bit. Thoughts @georgegevoian @jarek ?

[georgegevoian]

That’s surprising. The default product shouldn’t be labeled as legacy, and I just checked it isn’t on a fresh account.

@Ovi could you check if you’ve set GRIST_DEFAULT_PRODUCT in your environment? The default behavior (when unset) is to use a plan type that has no limits or restrictions.

George

[Ovi]

@georgegevoian

@Ovi could you check if you’ve set GRIST_DEFAULT_PRODUCT in your environment? The default behavior (when unset) is to use a plan type that has no limits or restrictions.

I just checked, and I did not set that variable mainly because I did not understand its explanation. I did not understand where to find names of PRODUCTS.

GRIST_DEFAULT_PRODUCT
if set, this controls enabled features and limits of new sites. See names of PRODUCTS in Product.ts.

@paul-grist

GRIST_DEFAULT_EMAIL=me@domain.tld
@Ovi could you try setting this variable ^ again? Grist needs to know who the initial administrator is, and this is one way to set it.

I will do that and try again, thanks for the idea.

I will also set:

GRIST_ANON_PLAYGROUND=FALSE 
GRIST_FORCE_LOGIN=FALSE

If I understood their meaning correctly, the first option will disable the “playground” meaning no unwelcome guests can play with my grist instance and the second one will still allow me to use features like sharing docs publicly with guests without them having an account on my grist instance?

Btw. if you have any other suggestions, I am more than happy to wipe my instance and start fresh as I haven’t used it productively yet. As you can see I am still busy setting it up securely and properly.

I have two more questions, since it appears they have been overlooked earlier.

A) What happens when I use the above shown “add account”? Where does this account get created, and where does it get added? I mean is it tied to the authentication method I use? Will the account be on org, team site or workspace level?

Sorry about A) I missed you already addressing this. I’ll leave the question though people don’t get confused by the edit.

From the README, I think you could hide the option by making GRIST_HIDE_UI_ELEMENTS contain multiAccounts , though I haven’t checked this.

B) What exactly is the relationship between orgs, team sites, workspaces, and users in the self-hosted version?

Say I have a grist instance for personal use, and enable GRIST_SINGLE_ORG. Now there’s only one org, fine. If I give another person access via my OIDC login, he gets a user account on my grist instance in my org, right, or do I somehow need to create a grist account for him too?

Can we now both use our respective personal site, create teams and workspaces within those team sites as well as share access to documents/workspaces/teams independently?

[paul-grist]

@Ovi

If I understood their meaning correctly, the first option will disable the “playground” meaning no unwelcome guests can play with my grist instance and the second one will still allow me to use features like sharing docs publicly with guests without them having an account on my grist instance?

That’s correct. Your setting for the GRIST_FORCE_LOGIN flag is already the default.

@Ovi

Say I have a grist instance for personal use, and enable GRIST_SINGLE_ORG. Now there’s only one org, fine. If I give another person access via my OIDC login, he gets a user account on my grist instance in my org, right, or do I somehow need to create a grist account for him too?

Broadly, your OIDC setup handles authentication, meaning it will work with users to verify their identity, and then vouch for that to Grist. Within Grist, you can control authorization, meaning: what does the user have the right to do. That’s where the team site sharing setup is useful:

Sharing team sites - Grist Help Center

In the team site sharing setup, it is possible to authorize a user that your OIDC setup would not allow through. In that case, your OIDC setup takes precedence, since the user will just never reach Grist.
It is also perfectly normal to have someone that your OIDC setup recognizes and authenticates, but they are not actually authorized to access a team site.
By default, users do not have access to team sites until it is specifically authorized (or some material on the site is shared in a public and list-able way). There are some reasonable feature requests around this. ANCT contributed a bulk-add-user UI to speed things up for themselves.

@Ovi

Can we now both use our respective personal site, create teams and workspaces within those team sites as well as share access to documents/workspaces/teams independently?

If GRIST_SINGLE_ORG is set, then no, that flag constrains Grist to have a single team site, so personal sites simply cannot be reached I believe.

Coming back to the relationship between orgs, team sites, workspaces, and users in the self-hosted version:

• Orgs is an old synonym for team sites that survives in environment variable names and source code.
• Workspaces are like non-nestable folders. They contain documents, and each workspace is in a team site.
• For users, I’m hoping the sketch of the separation of authentication and authorization responsibilities above helps a bit. User authorization operates at the site, workspace, and doc level - each have a “Manage Users” option. There is default inheritance of authorization by nested resources such as docs within workspaces within team sites. This can be overridden.
  There’s not much that is special to say about authorization, the support documentation should hold for self-hosted in large part, apart from stray references to SaaS limits.

[Ovi]

Well thanks for all that input and explanations. I tested 2 scenarios:

A) using GRIST_SINGLE_ORG and OIDC

This worked out exactly as expected, the email provided via GRIST_DEFAULT_EMAIL had admin access.I was now running a single org or team site to which I could add workspaces and invite team members to give access to.
I logged in as a second user and as expected OIDC let me through, but grist said: you have no access.

B) I did NOT set GRIST_SINGLE_ORG and used GRIST_ORG_IN_PATH as well as GRIST_DEFAULT_EMAIL.

The OIDC user with that email had admin access as expected. When I visit grist with this user it looks this way:

Seems fine to me as /o/docs/ looks like the default, the personal site works (no legacy shown any more) there is a workspace and a file I created.

Now switching to an anonymous browser with a second OIDC user:

Looks great, the workspace separation seems to work perfectly. Apparently any user I let through via OIDC, gets his own personal site?

Next I tried creating a team site. Seemed to work but threw an error:

Docker Logs show this error:

		 2024-06-07 10:11:58.596 - [33mwarn[39m: client error stack=TypeError: Cannot read properties of null (reading 'callback')                                │
│       at s.callAll (https://sheets.mydomain.tld/v/unknown/main.bundle.js:2:1057910)                                                                           │
│       at i.emit (https://sheets.mydomain.tld/v/unknown/main.bundle.js:2:1057437)                                                                              │
│       at S.setAndTrigger (https://sheets.mydomain.tld/v/unknown/main.bundle.js:2:1061444)                                                                     │
│       at S.set (https://sheets.mydomain.tld/v/unknown/main.bundle.js:2:1061361)                                                                               │
│       at u (https://sheets.mydomain.tld/v/unknown/main.bundle.js:2:856073), message=Cannot read properties of null (reading 'callback'), docId=undefined, p   │
│   age=https://sheets.mydomain.tld/o/docs/ws/2/, language=en-GB, platform=Win32, userAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHT   │
│   ML, like Gecko) Chrome/125.0.0.0 Safari/537.36, org=docs, email=ovidiu@mydomain.tld, userId=5, altSessionId=uEmN66PXvMH7SBz9G6GDgB                          │

The created team site works as expected. My second user had no access, upon invite he did. After I removed his access, he gets this message when still trying to access that team site:

The add account link, leads to ma authentik OIDC, and back to the same error message as expected because I have not customized any user creation flows on authentik.

I think this link to “add account” as well as the one in the next screenshot which btw both users see, not only the admin account could be hidden as explained above although I haven’t tested it yet

GRIST_HIDE_UI_ELEMENTS contain multiAccounts

Summary:
Perfect, I have figured everything out and it works as expected. Thanks everyone for your help. Btw. I will not use GRIST_SINGLE_ORG so that I can give i.e. my brother access to use my grist instance, while both having perfectly separate files/workspaces by default.

Bonus questions:

A) How do I delete a team site?
B) When creating a team site, I simply entered test as the domain and got an error. Entering testetesttest worked so there seems to be a minimum length check?

C) When managing a team, its URL still points towards getgrist.com even though it is working perfectly fien with ym own domain.

[Scojet_Support]

Hi Ovi…currently we have Grist self hosted and we’d like to integrate Authentik. I have not found a clear cut guide for this implementation. Would you be able to share the instructions that you were able to use to complete the configuration?

[Ovi]

Hi there,

I can try. Let me ask a few questions first:

• did you get grist working without OIDC?
• is authentik working, except for OIDC?
• do you have grist behind a working reverse proxy or does your infrastructure differ from this?

You create a OAuth2/OpenID Provider in authentik, here is mine:

Then you create an app inside authentik and assign it the previously created OIDC provider, here is mine:

then click on the name of your new provider:

where you will find all the info you need for your env variables for the grist container. Btw. auth.domain.tld is the URL of my authentik and sheets.domain.tld is the URl of my grist instance

Then click on your applications name:

On the grist / docker side, I’d say you require these env variables. I used more, but these should be enough.

GRIST_DEFAULT_EMAIL #for testing this should be the same as the authentik user you assigned to the bind policy earlier
GRIST_OIDC_IDP_ISSUER
GRIST_OIDC_IDP_CLIENT_ID
GRIST_OIDC_IDP_CLIENT_SECRET

I’m happy to share the rest of my docker-compose.yml but first get thing working before tinkering with the other config.

Please let me know if you have questions or get an error somewhere along the procedure.

[Scojet_Support]

Yes Grist is working without OIDC. I previously tried integrating Authentik and ran into problems. I am going to try again and will post the results. I appreciate you taking the time to reply and I will have an update shortly.

[Scojet_Support]

This is truly amazing…thank you again for taking the time to put this together. I received an error regarding a self signed certificate, and I suspect it’s because my Grist server does not have SSL installed. The Grist Omnibus instructions included a reference for “manual” SSL where you can specify your own SSL files but I cannot get the instructions to work. I have searched everywhere for guidance for SSL and I’ve seen variations of installs that include certbot or self encrypt. Are you using the non Enterprise version of Grist? If so, are you using SSL (your own certificates and not Lets Encrypt)? Or does Grist really only support auto generated SSL configurations. If you have information on your SSL install I would greatly appreciate it.

[Ovi]

You’re welcome.

To be honest, I never used Grist Omnibus.
I have a working traefik which I use for all my self-hosted docker containers and its easy adding an additional container to it hence I went with grist-core and used my existing traefik.

Traefik handles all my SSL needs by regularly getting me certificates from letsencrypt.

I can’t really share my traefik configs, as those have grown over the years of usage and are heavily tweaked, and it’s still traefik v2 while v3 was just released so it doesn’t make much sense.

Earlier you said:

Yes Grist is working without OIDC.

Do you mean its working behind your reverse proxy and SSL certificates are working too?
In that case, where did you get the error about the self-signed certificate, in a browser or in some logs? For which URL?

[Scojet_Support]

No thank you that’s actually very helpful. I am going to look at traefik. I initially only had the Grist self hosted installed (without SSL or a reverse proxy). When I completed the Authentik I got an error relating to a self signed certificate, and I thought that it was because my server doesn’t have a certificate installed which makes sense. I’ll recreate and post the error, but your information has already gotten me 99% farther than I have before so I truly appreciate the feedback…

[Scojet_Support]

I have made alot of progress. As of now I have Grist + SSL + traefik reverse proxy + Authentik almost fully working. The problem is, when I login to Authentik, select the Grist app, I am taken back to a “not found” page. I have tried variations of everything but can’t seem to get it right. This is the section for my Grist environment from my docker compose file. When you have time would you mind reviewing? I have 2 separate servers. One server for Authentik, and one for Grist + SSL + traefik.

grist:
image: gristlabs/grist
environment:
GRIST_DEFAULT_EMAIL: [email]
GRIST_SINGLE_ORG: grist
GRIST_ANON_PLAYGROUND: ‘FALSE’
APP_HOME_URL: https://gristserver.domain.com
GRIST_OIDC_SP_HOST: https://authentikserver.domain.com
GRIST_SESSION_SECRET: [secret]
GRIST_OIDC_IDP_ISSUER: https://authentikserver.domain.com/application/o/grist/
GRIST_OIDC_IDP_CLIENT_ID: [id]
GRIST_OIDC_IDP_CLIENT_SECRET: [secret]
GRIST_SAML_IDP_LOGIN: https://authentikserver.domain.com/application/saml/grist/sso/binding/redirect/
GRIST_SAML_IDP_LOGOUT: https://authentikserver.domain.com/if/session-end/grist/

[Scojet_Support]

Ok here is where I am…I can’t get OIDC to work with Authentik. I receive an error “not found” which resembles this OIDC providers not working after 2024.2.2 update - manual change required · Issue #8816 · goauthentik/authentik · GitHub. But I have not found a resolution. I tried the authentication methods here SAML - Grist Help Center, but I think the issue is that there isn’t a documented docker-compose file that lists the environment variables to properly tie everything together. If you would be willing to share your compose file (even a high level overview of the environment settings used for traefik + authentik + grist so that i can have an idea of what should be configured), I would greatly appreciate it as Grist’s documentation is very lacking.

[Ovi]

My first guess would be that you haven’t set the Redirect URL as shown in this screenshot. Despite authentik claiming you can leave it empty, that didn’t work with grist for me, I had to manually fill it in as shown.

[Ovi]

The problem is, when I login to Authentik, select the Grist app, I am taken back to a “not found” page.

IU never tried it that way, I visit grist, click login with OIDC, get redirected to authentik, log in, get redirected to grist.

What does the URL structure of the “not found” page look like?

Here is my complete env part of the compose file.

    environment:
      - TZ=Europe/Berlin
      - GRIST_DEFAULT_EMAIL=me@domain.tld
      - GRIST_OIDC_IDP_ISSUER=https://auth.domain.tld/application/o/grist/.well-known/openid-configuration
      - GRIST_OIDC_IDP_CLIENT_ID=secret
      - GRIST_OIDC_IDP_CLIENT_SECRET=secret
#      - GRIST_OIDC_IDP_END_SESSION_ENDPOINT=https://sheets.domain.tld
      - DEBUG=0
#tells Grist to drop support for multiple "team sites" which could introduce some subdomain complications.
#turned off for now as I played with multiple teams
#      - GRIST_SINGLE_ORG=sheets
      - GRIST_ORG_IN_PATH=true
      - GRIST_HIDE_UI_ELEMENTS=billing,multiAccounts,templates,supportGrist #comma-separated list of UI features to disable.
      - APP_DOC_URL=https://sheets.domain.tld
      - APP_HOME_URL=https://sheets.domain.tld
#the ip address for the server to listen to
      - GRIST_HOST=0.0.0.0
      - GRIST_DOMAIN=sheets.domain.tld #might be unnecessary #in hosted Grist, Grist is served from subdomains of this domain. Defaults to "getgrist.com".
# set secret
      - GRIST_SESSION_SECRET=secret
# sandboxing but not sure if this works
      - GRIST_SANDBOX_FLAVOR=gvisor
      - ALLOWED_WEBHOOK_DOMAINS=https://auto.domain.tld
      - GRIST_EXPERIMENTAL_PLUGINS=true
      - REDIS_URL=redis://grist-redis/0
      - GRIST_PAGE_TITLE_SUFFIX=" - Sheets"
      - GRIST_ANON_PLAYGROUND=FALSE #When set to 'false' deny anonymous users access to the home page
      - GRIST_FORCE_LOGIN=FALSE #Much like GRIST_ANON_PLAYGROUND but don't support anonymous access at all (features like sharing docs publicly requires aut>
#      - GRIST_SNAPSHOT_TIME_CAP= #optional. Define the caps for tracking buckets. Usage: {"hour": 25, "day": 32, "isoWeek": 12, "month": 96, "year": 1000}
      - GRIST_SNAPSHOT_KEEP=7 #optional. Number of recent snapshots to retain unconditionally for a document, regardless of when they were made
      - GRIST_BACKUP_DELAY_SECS=300 #wait this long after a doc change before making a backup
#      - GRIST_TEMPLATE_ORG= #set to an org "domain" to show public docs from that org
      - OPENAI_API_KEY=secret #optional. Synonym for ASSISTANT_API_KEY that assumes an OpenAI endpoint is>
      - ASSISTANT_API_KEY=secret #optional. An API key to pass when making requests to an external AI con>
#      - ASSISTANT_CHAT_COMPLETION_ENDPOINT= #optional. A chat-completion style endpoint to call. Not needed if OpenAI is being used.
#      - ASSISTANT_MODEL= #optional. If set, this string is passed along in calls to the AI conversational endpoint.
#      - ASSISTANT_LONGER_CONTEXT_MODEL= #optional. If set, requests that fail because of a context length limitation will be retried with this model set.

[Scojet_Support]

I did originally have the redirect url set but I think the issue is with my compose file. You mentioned a “login with OIDC” option within Grist and I did not have that option. That tells me my config isn’t right. I’m going to set my compose file like yours to see if that resolves. Thank you.

[Ovi]

Sorry, that was me being imprecise, the exact text is actually “sign in” - forget about login with OIDC. I remembered wrong, as I’m basically never signed out except if I open an incognito chrome tab.

[Scojet_Support]

I made 2 changes and this worked for me. Thank you SOOOOOOOOOOOOO much. I have spent so many hours/days working on this, and there is no way I would have figured this out.
*I had to add quotes around the true/false values
*I commented out REDIS_URL: and ALLOWED_WEBHOOK_DOMAINS
I appreciate the time you took to reply. Thank you again.