I created a grist with grist-electoron on Win11.
Next, I loaded the grist file with docker on Rasberry pi 400.
This reading did not go well and only one table called GRISTHIDDEN_IMPORT was created.
How to convert this GRISTHIDDEN_IMPORT to regular table?
Thanks in advance.
Hi @naoya.takahashi.
Just to confirm, you imported a .grist file, and not a .csv or .xlsx file? Only the latter should create tables with names like GristHidden_import.
George
What I read is a grist file.
“a” grist file is loaded, and no files other than GRISTHIDDEN_IMPORT is created.
What should I do?
One thing you can check is the contents of the file, to verify what tables are actually inside it. A .grist file is just a SQLite3 database, so any tool that can open it and show you the tables inside will work for this step. You can also try importing it on our hosted service if you’d prefer, although I expect you’ll see the same thing as your local instance of Grist. If you only see a GristHidden_import table in the Grist document, then something went wrong during the export from grist-electron. Does it still fail to import if you repeat the export from grist-electron?
George
Hi George.
I am sorry.
An error occurred at the time of grist-electron.
What should I do?
We can look at the debug output. If you’re running grist-electron from a terminal, you should already see it there. If not, you can create an empty file at ~/grist_debug.log and it’ll be populated with the output.
Sandboxing is enabled by default. It’s worth trying to run grist-electron without it enabled, in case there’s a problem with it on your machine. Setting GRIST_SANDBOX_FLAVOR=unsandboxed disables it.
I’d suggest opening an issue here now that we’ve isolated the problem to grist-electron.
George
There are txt and xlsx files created from Access.
When I tried to read this on Rasberry pi 400’s docker, an error occurred.
There seems to be a problem with grist-core.
----txt-----
2024-03-04 02:49:28.698 - warn: Error during api call to /workspaces/2/import: Failed to parse .txt file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready path=/workspaces/2/import, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718, uploadId=0, timezone=Asia/Tokyo,
2024-03-04 02:49:28.781 - debug: Auth[POST]: localhost:8484 /log customHostSession=, method=POST, host=localhost:8484, path=/log, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-04 02:49:28.786 - warn: client error stack=Error: Request to http://localhost:8484/dw/self/v/unknown/o/docs/api/workspaces/2/import failed with status 400: Bad Request (Failed to parse .txt file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready)
at y (http://localhost:8484/v/unknown/main.bundle.js:2:508688)
at O.request (http://localhost:8484/v/unknown/main.bundle.js:2:508408)
at async b.countPendingRequest (http://localhost:8484/v/unknown/main.bundle.js:2:507109)
at async O.requestJson (http://localhost:8484/v/unknown/main.bundle.js:2:508459)
at async b.countPendingRequest (http://localhost:8484/v/unknown/main.bundle.js:2:507109)
at async Tt (http://localhost:8484/v/unknown/main.bundle.js:2:159485)
at async http://localhost:8484/v/unknown/main.bundle.js:2:167991
at async Gt (http://localhost:8484/v/unknown/main.bundle.js:2:167889), message=Request to http://localhost:8484/dw/self/v/unknown/o/docs/api/workspaces/2/import failed with status 400: Bad Request (Failed to parse .txt file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready), status=400, tips=[action=ask-for-help, message=Ask for help], userError=Failed to parse .txt file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready, docId=undefined, page=http://localhost:8484/o/docs/ws/2/, language=ja, platform=Linux x86_64, userAgent=Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
----xlsx----
2024-03-04 02:57:06.799 - warn: Error during api call to /workspaces/2/import: Failed to parse Excel file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready path=/workspaces/2/import, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718, uploadId=1, timezone=Asia/Tokyo,
2024-03-04 02:57:06.857 - debug: Auth[POST]: localhost:8484 /log customHostSession=, method=POST, host=localhost:8484, path=/log, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-04 02:57:06.859 - warn: client error stack=Error: Request to http://localhost:8484/dw/self/v/unknown/o/docs/api/workspaces/2/import failed with status 400: Bad Request (Failed to parse Excel file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready)
at y (http://localhost:8484/v/unknown/main.bundle.js:2:508688)
at O.request (http://localhost:8484/v/unknown/main.bundle.js:2:508408)
at async b.countPendingRequest (http://localhost:8484/v/unknown/main.bundle.js:2:507109)
at async O.requestJson (http://localhost:8484/v/unknown/main.bundle.js:2:508459)
at async b.countPendingRequest (http://localhost:8484/v/unknown/main.bundle.js:2:507109)
at async Tt (http://localhost:8484/v/unknown/main.bundle.js:2:159485)
at async http://localhost:8484/v/unknown/main.bundle.js:2:167991
at async Gt (http://localhost:8484/v/unknown/main.bundle.js:2:167889), message=Request to http://localhost:8484/dw/self/v/unknown/o/docs/api/workspaces/2/import failed with status 400: Bad Request (Failed to parse Excel file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready), status=400, tips=[action=ask-for-help, message=Ask for help], userError=Failed to parse Excel file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready, docId=undefined, page=http://localhost:8484/o/docs/ws/2/, language=ja, platform=Linux x86_64, userAgent=Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
Are the files large? Do you see any mention of an out of memory error in the Docker logs?
George
I’m currently investigating.
Does loading fail when there are more than 60,000 lines…
This is xlsx import.
Thanks.
mo@raspberrypi:~ $ docker run -p 8484:8484 -v $PWD/persist:/persist -it gristlabs/grist
Welcome to Grist.
In quiet mode, see http://localhost:8484 to use.
For full logs, re-run with DEBUG=1
Setting up database…
Database setup complete.
2024-03-06 01:47:16.650 - info: == Grist version is 1.1.11 (commit unknown)
2024-03-06 01:47:16.672 - debug: skipping incomplete language ar (set GRIST_OFFER_ALL_LANGUAGES if you want it)
2024-03-06 01:47:16.675 - debug: skipping incomplete language cs (set GRIST_OFFER_ALL_LANGUAGES if you want it)
2024-03-06 01:47:16.714 - debug: skipping incomplete language fa (set GRIST_OFFER_ALL_LANGUAGES if you want it)
2024-03-06 01:47:16.760 - debug: skipping incomplete language nl (set GRIST_OFFER_ALL_LANGUAGES if you want it)
2024-03-06 01:47:16.821 - debug: skipping incomplete language th (set GRIST_OFFER_ALL_LANGUAGES if you want it)
2024-03-06 01:47:16.887 - info: Loading empty config because /persist/config.json missing
2024-03-06 01:47:16.928 - warn: did not find an appropriately named example workspace in deployment
2024-03-06 01:47:16.941 - info: No plugins found in directory: /grist/.grist/plugins
2024-03-06 01:47:16.973 - info: Found 2 valid plugins on the system
2024-03-06 01:47:16.975 - debug: PLUGIN builtIn/core – /grist/plugins/core
2024-03-06 01:47:16.977 - debug: PLUGIN bundled/grist-bundled – /grist/node_modules/@gristlabs/grist-widget/dist/plugins/grist-bundled
2024-03-06 01:47:17.004 - info: Server timeouts: keepAliveTimeout 305000 headersTimeout 306000
2024-03-06 01:47:17.019 - info: server(home,docs,static) available at 0.0.0.0:8484
2024-03-06 01:47:17.070 - info: activity docCount=17, orgCount=0, orgInGoodStandingCount=0, userCount=5, userWithLoginCount=5
2024-03-06 01:47:17.107 - warn: Failed to create GoogleAuth endpoint: GOOGLE_CLIENT_SECRET is not defined
2024-03-06 01:47:17.127 - info: Server timeouts: keepAliveTimeout 305000 headersTimeout 306000
2024-03-06 01:47:17.128 - info: pluginServer available at 0.0.0.0:43485
2024-03-06 01:47:17.137 - info: == appRoot: /grist
2024-03-06 01:47:17.138 - info: == i18:namespace: client,server
2024-03-06 01:47:17.139 - info: == docsRoot: /persist/docs
2024-03-06 01:47:17.140 - info: == defaultBaseDomain: undefined
2024-03-06 01:47:17.140 - info: == instanceRoot: /persist
2024-03-06 01:47:17.142 - info: == tag: unknown
2024-03-06 01:47:17.142 - info: == database: sqlite:///persist/home.sqlite3
2024-03-06 01:47:17.143 - info: == userRoot: /grist/.grist
2024-03-06 01:47:17.143 - info: == bundledRoot: /grist/node_modules/@gristlabs/grist-widget/dist
2024-03-06 01:47:17.143 - info: == loginMiddlewareComment: no-logins
2024-03-06 01:47:17.144 - info: == docWorkerId: testDocWorkerId_8484
2024-03-06 01:47:17.145 - info: == pluginUrl: http://0.0.0.0:43485/
2024-03-06 01:47:17.146 - info: == willServePlugins: true
2024-03-06 01:47:17.149 - info: == grist.access.supportEmail: support@getgrist.com [default] [GRIST_SUPPORT_EMAIL]
2024-03-06 01:47:17.149 - info: == grist.access.listPublicSites: false [default] [GRIST_LIST_PUBLIC_SITES]
2024-03-06 01:47:17.149 - info: == grist.integrations.sql.timeout: 1000 [default] [GRIST_SQL_TIMEOUT_MSEC]
2024-03-06 01:47:17.150 - info: == grist.integrations.allowedWebhookDomains: - [ALLOWED_WEBHOOK_DOMAINS]
2024-03-06 01:47:17.150 - info: == grist.integrations.proxy: - [GRIST_HTTPS_PROXY]
2024-03-06 01:47:17.150 - info: == grist.locale.offerAllLanguages: - [GRIST_OFFER_ALL_LANGUAGES]
2024-03-06 01:47:17.150 - info: == grist.login.system.forwardAuth.header: - [GRIST_FORWARD_AUTH_HEADER]
2024-03-06 01:47:17.151 - info: == grist.login.skipSession: - [GRIST_IGNORE_SESSION]
2024-03-06 01:47:17.151 - info: == grist.login.forced: - [GRIST_FORCE_LOGIN]
2024-03-06 01:47:17.151 - info: == grist.externalStorage.minio.bucket: - [GRIST_DOCS_MINIO_BUCKET]
2024-03-06 01:47:17.151 - info: == grist.externalStorage.disable: - [GRIST_DISABLE_S3]
2024-03-06 01:47:17.152 - info: == grist.externalStorage.active: false
2024-03-06 01:47:29.973 - debug: Auth[GET]: localhost:8484 /session/access/active customHostSession=, method=GET, host=localhost:8484, path=/session/access/active, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:47:30.069 - debug: Auth[GET]: localhost:8484 /session/access/all customHostSession=, method=GET, host=localhost:8484, path=/session/access/all, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:47:30.277 - debug: Auth[GET]: localhost:8484 /orgs/0/workspaces customHostSession=, method=GET, host=localhost:8484, path=/orgs/0/workspaces, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:47:30.400 - debug: Auth[GET]: localhost:8484 /orgs/0/usage customHostSession=, method=GET, host=localhost:8484, path=/orgs/0/usage, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:47:39.872 - debug: Auth[POST]: localhost:8484 /docs/4xbBvFw9hvZup3sUMFcZrd/remove customHostSession=, method=POST, host=localhost:8484, path=/docs/4xbBvFw9hvZup3sUMFcZrd/remove, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:47:40.167 - debug: Auth[GET]: localhost:8484 /orgs/0/workspaces customHostSession=, method=GET, host=localhost:8484, path=/orgs/0/workspaces, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:49:47.136 - debug: Auth[GET]: localhost:8484 /worker/import customHostSession=, method=GET, host=localhost:8484, path=/worker/import, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:49:47.211 - debug: Auth[POST]: localhost:8484 /uploads customHostSession=, method=POST, host=localhost:8484, path=/uploads, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:49:47.241 - debug: Prepared to receive upload into tmp dir /tmp/grist-upload-18P6186tYnG44 org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:49:48.776 - debug: Received file ANKEITO_TABLE_WORK (2).xlsx (40396097 bytes) org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:49:48.780 - debug: Created uploadId 0 in tmp dir /tmp/grist-upload-18P6186tYnG44 org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:49:48.831 - debug: Auth[POST]: localhost:8484 /workspaces/2/import customHostSession=, method=POST, host=localhost:8484, path=/workspaces/2/import, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:49:49.103 - debug: DocManager._createNewDoc picked name 2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:49.185 - debug: ActiveDoc loadDoc access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:49.187 - debug: ActiveDoc createDoc access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:49.258 - debug: DB 2cKLTwDpqn9eiKDakKj4r2 open successfully
2024-03-06 01:49:49.290 - debug: ActiveDoc Loading 22 normal tables, skipping 0 on-demand tables access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:49.291 - debug: ActiveDoc Normal tables: _grist_ACLMemberships, _grist_ACLPrincipals, _grist_ACLResources, _grist_ACLRules, _grist_Attachments, _grist_Cells, _grist_DocInfo, _grist_External_database, _grist_External_table, _grist_Filters, _grist_Imports, _grist_Pages, _grist_REPL_Hist, _grist_Shares, _grist_TabBar, _grist_TabItems, _grist_TableViews, _grist_Triggers, _grist_Validations, _grist_Views, _grist_Views_section, _grist_Views_section_field access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:49.291 - debug: ActiveDoc On-demand tables: access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:49.293 - debug: ActiveDoc starting to load 22 tables: _grist_ACLMemberships, _grist_ACLPrincipals, _grist_ACLResources, _grist_ACLRules, _grist_Attachments, _grist_Cells, _grist_DocInfo, _grist_External_database, _grist_External_table, _grist_Filters, _grist_Imports, _grist_Pages, _grist_REPL_Hist, _grist_Shares, _grist_TabBar, _grist_TabItems, _grist_TableViews, _grist_Triggers, _grist_Validations, _grist_Views, _grist_Views_section, grist_Views_section_field access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:49.377 - debug: 3-pipe Sandbox started sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:49.420 - debug: UploadSet: moving uploadId 0 to /tmp/grist-tmp-1CiqBaqBdCq1g
2024-03-06 01:49:49.431 - info: ActiveDoc.importFileAsNewTable(/tmp/grist-tmp-1CiqBaqBdCq1g/grist-upload-187emAVaTZ3MT/sijX0v1VWBbzYNGne0kzchO.xlsx, ANKEITO_TABLE_WORK (2).xlsx)
2024-03-06 01:49:49.435 - debug: parseFile: found 1 fileParser with matching file extensions
2024-03-06 01:49:49.435 - info: DocPluginManager.parseFile: calling to builtIn/core with grist-upload-187emAVaTZ3MT/sijX0v1VWBbzYNGne0kzchO.xlsx
2024-03-06 01:49:49.437 - debug: PLUGIN builtIn/core: Rpc for xls_parser.parseFile#1: RPC_CALLING
2024-03-06 01:49:49.438 - debug: PLUGIN builtIn/core: Rpc sending RpcCall: xls_parser.parseFile#1
2024-03-06 01:49:49.439 - debug: PLUGIN builtIn/core: Rpc for xls_parser.parseFile#1: RPC_ONCALL
2024-03-06 01:49:49.441 - debug: PLUGIN builtIn/core SafePython: Activating plugin component
2024-03-06 01:49:49.453 - debug: 3-pipe Sandbox started sandboxPid=19, flavor=unsandboxed, command=undefined, entryPoint=(default), plugin=builtIn/core, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:51.985 - info: Sandbox stderr: [INFO] [main] Ready sandboxPid=19, flavor=unsandboxed, command=undefined, entryPoint=(default), plugin=builtIn/core, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:51.986 - info: Sandbox stderr: [INFO] [main] Ready sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:52.309 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_meta_tables, loadMs=2930
2024-03-06 01:49:52.317 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=6
2024-03-06 01:49:52.319 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=6
2024-03-06 01:49:52.323 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=7
2024-03-06 01:49:52.323 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=8
2024-03-06 01:49:52.330 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=13
2024-03-06 01:49:52.330 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=5
2024-03-06 01:49:52.331 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=6
2024-03-06 01:49:52.332 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=6
2024-03-06 01:49:52.332 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=6
2024-03-06 01:49:52.338 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=4
2024-03-06 01:49:52.339 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=4
2024-03-06 01:49:52.339 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=4
2024-03-06 01:49:52.340 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=5
2024-03-06 01:49:52.341 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=6
2024-03-06 01:49:52.346 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=4
2024-03-06 01:49:52.349 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=7
2024-03-06 01:49:52.350 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=7
2024-03-06 01:49:52.351 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=8
2024-03-06 01:49:52.352 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=7
2024-03-06 01:49:52.354 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=7
2024-03-06 01:49:52.356 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=4
2024-03-06 01:49:52.356 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=load_table, loadMs=3
2024-03-06 01:49:52.358 - debug: ActiveDoc loading 0 tables: access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:52.367 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=get_table_stats, loadMs=2
2024-03-06 01:49:52.368 - info: Loading complete, table statistics retrieved… access=owners, docId=2cKLTwDpqn9eiKDakKj4r2, num_user_tables=0, _grist_Tables_rows=0, _grist_Tables_bytes=161, _grist_Tables_column_rows=0, _grist_Tables_column_bytes=325, _grist_ACLMemberships_rows=0, _grist_ACLMemberships_bytes=45, _grist_ACLPrincipals_rows=4, _grist_ACLPrincipals_bytes=270, _grist_ACLResources_rows=1, _grist_ACLResources_bytes=62, _grist_ACLRules_rows=1, _grist_ACLRules_bytes=280, _grist_Attachments_rows=0, _grist_Attachments_bytes=188, _grist_Cells_rows=0, _grist_Cells_bytes=143, _grist_DocInfo_rows=1, _grist_DocInfo_bytes=211, _grist_External_database_rows=0, _grist_External_database_bytes=112, _grist_External_table_rows=0, _grist_External_table_bytes=72, _grist_Filters_rows=0, _grist_Filters_bytes=86, _grist_Imports_rows=0, _grist_Imports_bytes=219, _grist_Pages_rows=0, _grist_Pages_bytes=87, _grist_REPL_Hist_rows=0, _grist_REPL_Hist_bytes=67, _grist_Shares_rows=0, _grist_Shares_bytes=83, _grist_TabBar_rows=0, _grist_TabBar_bytes=47, _grist_TabItems_rows=0, _grist_TabItems_bytes=49, _grist_TableViews_rows=0, _grist_TableViews_bytes=49, _grist_Triggers_rows=0, _grist_Triggers_bytes=138, _grist_Validations_rows=0, _grist_Validations_bytes=63, _grist_Views_rows=0, _grist_Views_bytes=62, _grist_Views_section_rows=0, _grist_Views_section_bytes=390, _grist_Views_section_field_rows=0, _grist_Views_section_field_bytes=176, num_on_demand_tables=0
2024-03-06 01:49:52.370 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=initialize, loadMs=1
2024-03-06 01:49:52.377 - debug: ActiveDoc _applyUserActions(null, [[‘Calculate’]]) access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:52.403 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=apply_user_actions, loadMs=13
2024-03-06 01:49:52.476 - debug: Sandbox memory memory=51175424, sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:52.483 - debug: Sharing _doApplyUserActions access=owners, docId=2cKLTwDpqn9eiKDakKj4r2, actionNum=0, linkId=0, otherId=0, numDocActions=0, numRows=0, author=grist
2024-03-06 01:49:52.497 - info: Sandbox row count access=owners, docId=2cKLTwDpqn9eiKDakKj4r2, rowCount=0
2024-03-06 01:49:52.508 - debug: ActiveDoc _applyUserActions returning {actionNum: 0, retValues: [null], isModification: false} access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:52.510 - debug: ActiveDoc load timing access=owners, docId=2cKLTwDpqn9eiKDakKj4r2, loadMs=3325, closeTimeout=16625
2024-03-06 01:49:52.511 - info: ActiveDoc fetchMetaTables access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:49:59.458 - warn: Slow pyCall sandboxPid=19, flavor=unsandboxed, command=undefined, entryPoint=(default), plugin=builtIn/core, docId=2cKLTwDpqn9eiKDakKj4r2, funcName=xls_parser.parseFile
2024-03-06 01:54:46.903 - info: Data size from dbstat… access=owners, docId=2cKLTwDpqn9eiKDakKj4r2, dataSizeBytes=386, timeToMeasure=1
2024-03-06 01:55:06.052 - debug: ActiveDoc inactivity timeout docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:55:06.054 - debug: ActiveDoc shutdown starting access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:55:06.063 - debug: ActiveDoc _applyUserActions(null, [[‘RemoveTransformColumns’]]) access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:55:06.067 - debug: NSandbox pyCall sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2, funcName=apply_user_actions, loadMs=2
2024-03-06 01:55:06.071 - debug: Sandbox memory memory=51175424, sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:55:06.072 - debug: Sharing _doApplyUserActions access=owners, docId=2cKLTwDpqn9eiKDakKj4r2, actionNum=0, linkId=0, otherId=0, numDocActions=0, numRows=0, author=grist
2024-03-06 01:55:06.076 - info: Sandbox row count access=owners, docId=2cKLTwDpqn9eiKDakKj4r2, rowCount=0
2024-03-06 01:55:06.077 - debug: ActiveDoc _applyUserActions returning {actionNum: 0, retValues: [null], isModification: false} access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:55:06.093 - debug: DocPluginManager.shutdown cleaning up 2 plugins
2024-03-06 01:55:06.096 - debug: PLUGIN builtIn/core SafePython: Deactivating plugin component
2024-03-06 01:55:06.097 - info: SafePython deactivating …
2024-03-06 01:55:06.098 - debug: Sandbox shutdown starting sandboxPid=19, flavor=unsandboxed, command=undefined, entryPoint=(default), plugin=builtIn/core, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:55:06.099 - debug: Sandbox shutdown starting sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:55:06.100 - debug: DocStorage shutdown success
2024-03-06 01:55:06.760 - debug: Sandbox exited with code 0 signal null sandboxPid=18, flavor=unsandboxed, command=undefined, entryPoint=(default), docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:55:07.100 - warn: Sandbox sending SIGKILL sandboxPid=19, flavor=unsandboxed, command=undefined, entryPoint=(default), plugin=builtIn/core, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:55:07.177 - debug: NSandbox pyCall sandboxPid=19, flavor=unsandboxed, command=undefined, entryPoint=(default), plugin=builtIn/core, docId=2cKLTwDpqn9eiKDakKj4r2, funcName=xls_parser.parseFile, loadMs=317721
2024-03-06 01:55:07.179 - warn: PLUGIN builtIn/core: Rpc for xls_parser.parseFile#1: RPC_ONCALL_ERROR [Sandbox] [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready
2024-03-06 01:55:07.179 - debug: PLUGIN builtIn/core: Rpc sending RpcRespErr
2024-03-06 01:55:07.180 - debug: PLUGIN builtIn/core: Rpc for xls_parser.parseFile#1: RPC_RESULT_ERROR [Sandbox] [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready
2024-03-06 01:55:07.181 - warn: DocPluginManager.parseFile: builtIn/core Failed parseFile [Sandbox] [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready
2024-03-06 01:55:07.182 - debug: UploadSet: cleaning up uploadId 0
2024-03-06 01:55:07.184 - info: SafePython done deactivating the sandbox
2024-03-06 01:55:07.185 - debug: DocPluginManager.shutdown removing tmpDir /tmp/grist-tmp-1CiqBaqBdCq1g
2024-03-06 01:55:07.186 - debug: Sandbox exited with code null signal SIGKILL sandboxPid=19, flavor=unsandboxed, command=undefined, entryPoint=(default), plugin=builtIn/core, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:55:07.200 - debug: ActiveDoc shutdown complete access=owners, docId=2cKLTwDpqn9eiKDakKj4r2
2024-03-06 01:55:07.232 - warn: Error during api call to /workspaces/2/import: Failed to parse Excel file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready path=/workspaces/2/import, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718, uploadId=0, timezone=Asia/Tokyo,
2024-03-06 01:55:07.305 - debug: Auth[POST]: localhost:8484 /log customHostSession=, method=POST, host=localhost:8484, path=/log, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
2024-03-06 01:55:07.308 - warn: client error stack=Error: Request to http://localhost:8484/dw/self/v/unknown/o/docs/api/workspaces/2/import failed with status 400: Bad Request (Failed to parse Excel file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready)
at y (http://localhost:8484/v/unknown/main.bundle.js:2:508688)
at O.request (http://localhost:8484/v/unknown/main.bundle.js:2:508408)
at async b.countPendingRequest (http://localhost:8484/v/unknown/main.bundle.js:2:507109)
at async O.requestJson (http://localhost:8484/v/unknown/main.bundle.js:2:508459)
at async b.countPendingRequest (http://localhost:8484/v/unknown/main.bundle.js:2:507109)
at async Tt (http://localhost:8484/v/unknown/main.bundle.js:2:159485)
at async http://localhost:8484/v/unknown/main.bundle.js:2:167991
at async Gt (http://localhost:8484/v/unknown/main.bundle.js:2:167889), message=Request to http://localhost:8484/dw/self/v/unknown/o/docs/api/workspaces/2/import failed with status 400: Bad Request (Failed to parse Excel file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready), status=400, tips=[action=ask-for-help, message=Ask for help], userError=Failed to parse Excel file.
Error: [Sandbox] PipeFromSandbox is closed: [INFO] [main] Ready, docId=undefined, page=http://localhost:8484/o/docs/ws/2/, language=ja, platform=Linux x86_64, userAgent=Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36, org=docs, email=you@example.com, userId=5, altSessionId=6K4UgTSnnDfqgyarNFa718
Thanks for the output.
Does the same file import successfully if you remove enough lines to get below 60,000?
There’s no hard limit that we enforce, but it’s certainly possible it’s too big for your machine (I’m assuming your Pi only has 4 GB). For moderate workloads, we’ve found 8 GB of ram to work well, though you can certainly get by with a lot less. It just depends on how big the documents you plan to use are, and how many of them you plan to simultaneously have open.
I don’t see any indication of memory issues in this particular case, though it does look like it’s hanging shortly after starting import.
George
Thanks @naoya.takahashi , support received your file.
It looks like what you are running into is the 5-minute timeout for parsing imports. The reason it’s taking so long is that you are running on Raspberry Pi, and the file is very large. It is actually at the limit of what we recommend even for a normal machine, see Limits - Grist Help Center (due to many rows and many columns).
If you can do the import on a faster machine, then you can download the complete .grist file (using the Download option in the Share menu), and import it on a different machine (even Raspberry Pi) much faster. Unlike Excel or CSV, importing a .grist file into Grist does not require any parsing.
Alternatively, you may be able to adjust the timeout given to the import parser by changing the file plugins/core/manifest.yml, and adding components.deactivate.inactivitySec value (under components, like this (the default is 300):
components:
safePython: sandbox/main.py
deactivate:
inactivitySec: 3600