Grist accepting data very slowly

1

I’m having to enter data very slowly into my app. I get the “working” message a lot.

Can you look at it to see if you can tell me the problem?

you already have access to the document

thanks

2

Hi Mike,

Hm, I suspect it’s some of these formulas. There may be ways to make them more efficient. Though I am not seeing a very slow performance. In the tables most heavily calculated with formulas, there’s a slight lag of about one second. Could you record the issue and share with support@getgrist.com ?

3

Also have a look at the raw data, there might be some huge tables that are unlinked but not deleted from the sqlite database. This also slows down grist quite a lot.
Also i’ve noticed a significant slowdown when i reach about ~100.000 rows.

4

There looks to be a stale and hidden helper column in the document that has a slow formula in it.

  • About helper columns. Helper columns are created in the background for various column types to help store extra information needed. We are unsure on what exactly led to the helper column being created in your document. One possibility: When changing a column type, reloading the page before applying or canceling the column type transformation.

  • Finding hidden helper columns via the API. https://$host/api/docs/$docid/tables/$tableid/columns?hidden=true

  • Deleting helper columns via the API Apply this change per helper column

curl -XDELETE -H "Authorization: Bearer $key" -H "Content-Type: application/json" https://$host/api/docs/$docid/tables/$tableid/columns/$columnid
5

thanks Anais

I deleted a bunch of formulas last week so things entered faster.

But it’s slowed down again so I was gonna check for helper columns.

I typed the link you send into chrome, but just got a search page.
https://$host/api/docs/$docid/tables/$tableid/columns?hidden=true

So I’m not sure how to proceed. Can you give me a little more insight into how to check for and remove the helper columns

thanks

6

I’m still having trouble with very slow data entry. Can you help me with how to remove helper columns.

7

Grist has now become unusable. It can take 17 seconds to accept an entry.

Is there any way you can help me?

8

Within the link @anais-grist gave you (https://$host/api/docs/$docid/tables/$tableid/columns?hidden=true), you have to replace $host by your domain name, and $tableid by the id of the table you want to inspect. That way, you should be able to know which columns are stale, especially those that contain heavy formulas.

9

thanks @jperon

I tired replacing the $host, $tableid and also I assumed $docid would need to be replaced with the document name since I have many documents.

I just got an error that said the site can’t be reached so I must be tying in something wrong.

I assume I replace $host with my domain, removing the $. is that correct?

10

It is. Let’s say your domain is example.com:

https://example.com/api/docs/7jHM0qpTHses/tables/TableName/columns?hidden=true

To know what the $docid is, you have to launch two requests before:

https://example.com/api/orgs/ will give you a list of organizations. Let’s say you have an exampleorganization organization:

https://example.com/api/orgs/exampleorganization/workspaces/ will give you a list of workspaces and docs. Then note the id (not name) of your document, and use it in the first url in place of 7jHM0qpTHses.

11

Hi @Mike_Moeller.

We made some improvements recently to automatically clean up helper columns when a document isn’t open. You can force a reload of your document by clicking Settings in the left panel, and then clicking the skull icon (see screenshot). Could you give that a shot and see if it helps with the slowdown?

Screenshot 2023-11-20 at 3.41.51 PM
Screenshot 2023-11-20 at 3.41.51 PM978×836 46.5 KB

George

12

it turns out the helper column wasn’t as big of a problem as my formulas. I have a lot of formulas that are summarizing data.

Until Grist adds an autocalc feature that I can turn off, I was wondering if another strategy might work.

I’m not very good in python, but if I took one of my complex formulas and wrapped in in an “'if statement”, wouldn’t that stop the complex calculating unless I toggled it to true?

I was thinking of adding a field to my global variable table called “CalcOn”. I’ll make it a toggle field. then if I create each of my formulas like this…

if !CalcOn
{
return “”
}
else
{
very complex formula
return “answer to complex formula”
}

it seems like if CalcOn is false, then Grist will have very little processing to do and will run fast.

Then I can toggle CalcOn and get my summaries.

before I go in and change all my formulas I was wondering if a python/Grist expert can let me know if this is likely to work.

13

I have a similar workaround, but which doesn’t work with summary tables: using a trigger formula, with a toggle column that triggers it when I want to update results. I think your trick would work; the only drawback I see is that toggling it to False will clear all results, so toggling back to True will trigger all calculations (instead of updating only what changed).

Depending on the type of your columns, perhaps you’d better return None instead of "". Moreover, conditions require no braces in Python, but indentation instead. So, assuming your global table is a key/value one:

if not MY_KEYVALUE_GLOBAL_TABLE.lookupOne(key="CalcOn").value:
    return
else
    very complex formula
    return “answer to complex formula”
2 Likes
14

I’m testing the logic I want to use and did something wrong.

I assume that the code I have should print the total when the if statement is true and Blank if its not

so when I type “if FALSE:”, it should run the else which is a Blank. But its calculating the total and returning that

What did I do wrong?

image

15

Use FALSE() or False.

1 Like
16

I also feel as though there’s been a noticeable slowdown over the past three months.

Is this something that could be improved if I moved to a self-hosted grist with higher specs?