New API feature request : POST grist file

Xavier_de_Rochefort · August 17, 2023, 7:56am

Hello everyone,

At now the Grist API allows to download a document in its grist format (sqlite) : GET /{docId}/download.

We would also like to be able to POST a grist document. Alternatively, it could also be a route to call the duplicate function that already exists in grist.

Our use case : we have a template file (with specific views, access rules etc) that we want to use as a base file when we create a new document.

Would it be conceivable to add this feature ? Does anyone would be also interested by this feature ?

Do I create a new issue on github about it ?

Thank you,

Xavier

PS : @dmitry-grist we could have a budget to see this feature added

paul-grist · August 17, 2023, 1:33pm

POSTing a grist document is just about possible through some undocumented heroics. It would definitely be great to have a simple, documented endpoint for this.

First, there is undocumented endpoint that almost works - you can POST a .grist file to /api/docs - but it will only give you an “unsaved” document, not something in a workspace:

github.com

gristlabs/grist-core/blob/1774d0314fd33d8a97151f428249398878a17c0e/app/server/lib/DocApi.ts#L1170-L1192


      
          // Create a document.  When an upload is included, it is imported as the initial
          // state of the document.  Otherwise a fresh empty document is created.
          // A "timezone" option can be supplied.
          // Documents are created "unsaved".
          // TODO: support workspaceId option for creating regular documents, at which point
          // existing import endpoint and doc creation endpoint can share implementation
          // with this.
          // Returns the id of the created document.
          this._app.post('/api/docs', expressWrap(async (req, res) => {
            const userId = getUserId(req);
            let uploadId: number|undefined;
            let parameters: {[key: string]: any};
            if (req.is('multipart/form-data')) {
              const formResult = await handleOptionalUpload(req, res);
              if (formResult.upload) {
                uploadId = formResult.upload.uploadId;
              }
              parameters = formResult.parameters || {};
            } else {
              parameters = req.body;

This file has been truncated. show original

The endpoint has a TODO to support specifying a workspace which, if implemented, would allow creating a regular document in a workspace. I suspect completing the TODO would not be difficult, it just wasn’t necessary for the specific job this was put in place for.

There is an alternative sequence of operations possible to POST a document to a workspace, using a pair of undocumented endpoints, /uploads and /api/workspaces/<workspace-id>/import. I see that sequence used in tests:

github.com

gristlabs/grist-core/blob/1774d0314fd33d8a97151f428249398878a17c0e/test/nbrowser/homeUtil.ts#L270-L280


      
          /**
           * Import a fixture doc into a workspace.
           */
          public async importFixturesDoc(username: string, org: string, workspace: string,
                                         filename: string, options: {newName?: string, email?: string} = {}) {
            const homeApi = this.createHomeApi(username, org, options.email);
            const docWorker = await homeApi.getWorkerAPI('import');
            const workspaceId = await this.getWorkspaceId(homeApi, workspace);
            const uploadId = await this.uploadFixtureDoc(docWorker, filename, options.newName);
            return docWorker.importDocToWorkspace(uploadId, workspaceId);
          }

This is basically the process used by the Grist web app when the user uploads a .grist file. It is way more convoluted than it needs to be, because of steps in how Grist evolved from a pure desktop app to something that may involve multiple servers.

Creating an issue on github would be a good start. There are two main teams developing Grist: Grist Labs, and ANCT, and we collaborate primarily via github issues. We are also getting a stream of contributions from other Grist users.

I think @georgegevoian was looking at something adjacent to this recently related to copying documents, and may be adding a simplification there - but I don’t know if that is touching direct uploads, is it @georgegevoian ?

georgegevoian · August 17, 2023, 4:17pm

Unfortunately, the simplification targets a slightly different use case: making a copy of an already-uploaded Grist document. But as you mentioned, the /docs endpoint is almost there, and I think it would make a good first issue.

paul-grist · August 17, 2023, 4:40pm

Agree. And the last issue that Florent tagged as a good-first-issue got picked up and implemented pretty quickly

github.com/gristlabs/grist-core

Support Ctrl-Shift-(arrow keys)

opened 08:24AM - 26 Jul 23 UTC

closed 04:21PM - 17 Aug 23 UTC

fflorent

good first issue anct

NB: this idea is part of issue #548, extracted from it so it's scope is defined …and so we can add `good-first-issue` tag On Excel and LibreOffice calc, the Ctrl-Shift-(arrow keys) shortcuts are quite useful as a selection tool. They allows to select from the cursor to select the rest of the cells to the direction of the arrow until either an empty cell or the end of the document is met. It would help if Grist could have the same shortcuts. https://github.com/gristlabs/grist-core/assets/371705/742bd4af-79d5-4d61-8fba-b9139a89c2c1

Xavier_de_Rochefort · August 21, 2023, 9:29am

Thank you for your answers @paul-grist and @georgegevoian. The issue has just been created : API enhancement : upload a new document in a workspace · Issue #635 · gristlabs/grist-core · GitHub

Xavier_de_Rochefort · August 30, 2023, 2:15pm

Hello everybody,

We would need it pretty quickly.

Waiting for the issue to be implemented, we are trying to use the /uploads and /workspaces/:wid/import approach. Can we call the /uploads endpoint with the same bearer token we use for other api calls or is it intended to be authenticated through an app session ?

Thank you !

Xavier_de_Rochefort · August 30, 2023, 2:24pm

Also, our use-case could be very well answered by an easy way to make a copy of our template, which is a grist document on the same instance at now.

georgegevoian · August 30, 2023, 10:43pm

Hi @Xavier_de_Rochefort.

I’ll look into adding workspaceId as a parameter of /api/docs this week. The approach with the 2 undocumented endpoints has some gotchas, so I wouldn’t recommend it.

George

Xavier_de_Rochefort · August 31, 2023, 8:25am

Thanks a lot @georgegevoian. Do you think that the docker image will be updated with that feature in the same time ?

georgegevoian · September 1, 2023, 7:26am

The latest tag gets updated daily, so as soon as this PR is reviewed and merged the next update should include it.

Xavier_de_Rochefort · September 1, 2023, 8:03am

This is good news ! Thanks a lot !

georgegevoian · September 5, 2023, 9:34pm

The next update to the image will include this feature. You just have to pass along workspaceId (either in the JSON body or form data), and (optionally) a documentName to the endpoint.

The same endpoint will soon also support duplicating a document, with an option to exclude all table data. (Essentially making it an API equivalent of the Duplicate Document dialog.)

Xavier_de_Rochefort · September 6, 2023, 11:42am

Thanks a lot @georgegevoian ! That is very good news for us.