Python HTTP for Humans as cross docs data exchange

BiBo · March 22, 2023, 8:26am

Hello, is there any plan to integrate the HTTP for Humans library into Grist?
Found this option for cross docs data exchange, is it possible to install external packages in self-hosted setup?

paul-grist · March 23, 2023, 6:35pm

There are some notes on installing extra python packages in self-hosted Grist here:

There is actually an undocumented, disabled REQUEST() function that works similarly to requests:

github.com

gristlabs/grist-core/blob/3faab1a03566dc68d6cb4cfa52538e5e60c77a93/sandbox/grist/functions/info.py#L661-L681


      
          def REQUEST(url, params=None, headers=None):
            # Makes a GET HTTP request with an API similar to `requests.get`.
            # Actually jumps through hoops internally to make the request asynchronously (usually)
            # while feeling synchronous to the formula writer.
          
            # Requests are identified by a string key in various places.
            # The same arguments should produce the same key so the request is only made once.
            args = dict(url=url, params=params, headers=headers)
            args_json = json.dumps(args, sort_keys=True)
            key = hashlib.sha256(args_json.encode()).hexdigest()
          
            # This may either return the raw response data or it may raise a special exception
            # to delegate the request and reevaluate the formula later.
            response_dict = docmodel.global_docmodel._engine._requesting(key, args)
          
            if "error" in response_dict:
              # Indicates a complete failure to make the request, such as a connection problem.
              # An unsuccessful status code like 404 or 500 doesn't raise this error.
              raise HTTPError(response_dict["error"])
          
            return Response(**response_dict)

You can use this method if you set GRIST_EXPERIMENTAL_PLUGINS=1 in Grist’s environment. As far as I remember, the syntax was:

response = REQUEST("...url...")
# should now have response.status_code, response.text, response.encoding, response.json()
response = REQUEST("...url...", params=dict(bar=123))  # to add query parameters conveniently

The function is disabled for a reason though, it is quite easy to make a mess using it, since cells can end up getting reevaluated for various reasons, which (when you use REQUEST) means requests get re-sent - maybe lots of them in a big doc. In trigger formulas, it could be useful. We didn’t figure out a way to communicate its limits well, and feared a big support load if we released it, so it got shelved. Apologies in advance, but I won’t be able to provide support if you use this function.