"Add new" -> "Add table from template"

Hi,

It would be really nice, probably mainly during the exploration and development stage of a Grist document, to be able to quickly add a new table with columns from some kind of table template or schema. Currently I am keeping such an empty table and duplicate it from the “Raw Data” view.

Perhaps it would be a small change to be able to flag some table in the raw data as “Template” and then have a choice “Add tebvle from template” under the “Add new” dropdown, where the user then can select from the raw data tables marked as templates.

Possibly one could even select multiple templates, so that one could be e.g. some particula set of columns for dim tables with standardized column names, another one audit columns, etc.

The audit columns are the biggest pain point for me currently: After adding a new table to the document, I have to click 8 times and navigate the context menu four times to get my four columns set up that I use in nearly all my tables.

Also, if this template flag would be implemented, one could also add columns from those “by hand" to an existing table, by selecting the appropriate template under a new entry in the “Shortcuts” section.

Looking forward to hearing your thoughts on this!

Simon

@SHK I’m not sure I understand what the use case is: Why do you want to create empty clones of a table inside a single document? Can you describe what you’re trying to do?

Sure thing, thanks for the question, @smaffulli .

Let’s take my current project where I am planning to use Grist as the tool to manage our Master Data, which will then furthermore be synced with our internal RDB.

I want to have the dimension tables as identical as possible in setup, to have the system as straightforward and understandable as possible, also for new users, and also make sure I have all fields available that I will need/do want for the import into our database.

The template I created below as an empty table has a lot of the default features/properties I want in my tables:

  • key: business key
  • name: Free text name
  • description
  • comment
  • label: To be used in Reference dropdowns, based on “name” with “key” as tiebreaker where needed.
  • sk: “id” as surrogate key for import
  • Audit columns: for creation and last change
  • Duplicate checks: for “label” creation and highlighting or blocking of duplicate keys.

By this, I can be sure that selecting the “label” field as the display of a reference column I will get meaningful and unique names, I have my audit columns set up (right from the start, before any data is entered or imported), and the “sk” column to be read to populate our internal RDB.

100% correct/expected field names, no danger of typos (“Key” vs “key”), no manual steps to create the same columns over and over again.

Once this minimum set of columns is set up, I can add anything else I need in there, or, e.g. add an extra column and change “key” to a formula column, to create the business keys automatically, without having to adjust anything downstream.

I hope that makes it a bit clearer. Please let me know if it is still incomprehensible.

@grist.UserTable
class Dim_table_template:
  key = grist.Text()
  name = grist.Text()
  description = grist.Text()
  comment = grist.Text()

  @grist.formulaType(grist.Bool())
  def Duplicate_in_key(rec, table):
    return rec.key != "" and rec.key is not None and len(Dim_table_template.lookupRecords(key=rec.key)) > 1

  @grist.formulaType(grist.Bool())
  def Duplicate_in_name(rec, table):
    return rec.name != "" and rec.name is not None and len(Dim_table_template.lookupRecords(name=rec.name)) > 1

  @grist.formulaType(grist.Text())
  def label(rec, table):
    if rec.Duplicate_in_name:
      return f'{rec.name} ({rec.key})'
    else:
      return rec.name

  @grist.formulaType(grist.Int())
  def sk(rec, table):
    return rec.id


  def _default_Created_at(rec, table, value, user):
    return NOW()
  Created_at = grist.DateTime('Atlantic/Reykjavik')

  def _default_Created_by(rec, table, value, user):
    return user.Name
  Created_by = grist.Text()

  def _default_Last_updated_at(rec, table, value, user):
    return NOW()
  Last_updated_at = grist.DateTime('Atlantic/Reykjavik')

  def _default_Last_updated_by(rec, table, value, user):
    return user.Name
  Last_updated_by = grist.Text()


I understand you’re trying to copy a table schema, but I don’t understand why you would want to do that with a relational database.

You mentioned that you have an external databaseinternal DB while Grist document is your Master Data . You also say that you have multiple users so you want them to add data to Master Data. I assume that such data will be moved from Master DB to internal DB at some point… At this point I get lost: I don’t see why you need to re-create the table again.

Can you describe your process in more details so we can understand it and suggest you another approach?

At the point where you get lost there is actually no need to recreate any table any more, so your intuition on that is correct. What I am describing is the process of getting to that point. And I can even take all outside of Grist out of the equation.

In cases where I have to create 5-15 dim tables of very similar structure, it would be nice to being able to start constructing them off from a template with common fields, than from scratch.

That is pretty much the core of it. I hope this clarifies.

The current approach is this template table that I have designed, which I then duplicate in the raw data view. The downside here is that I have to add the table widget to the Document manually afterwards, and adjust the Name after the page with the table widget had been created with its default name.

Let’s see if I get this right. Your need is for a way to create in a single Grist document one “master” table structure that you can replicate. You need to start from one common table structure, like:

ID Name Address City

That structure is stable and never (or very rarely) changes.

Then you want to create more tables based on that “master’“ structure but with slight modifications, like:

| ID | Name | Addresss | City | Origin |

| ID | Name | Addresss | City | Destination |

| ID | Name | Addresss | City | Foo |

| ID | Name | Addresss | City | Bar |

Is this what you have in mind? What do you do with these tables? Who needs to create them? How are Grist users going to use them?

I think you understood the core concept correctly. Addressing your statements and questions first, before showing a more concrete example on how this would look in my case.

Yes, I am trying to create one (or possibly more) “master” table structure, or “template” as I called it, that can be replicated to form a new table, to then be used as is, or modified by adding extra columns.

The structure should be stable, once one has defined the columns that should be appearing in the newly constructed tables. But it does not need to be required. (As the templates are just copied when needed, their result is completely detached from them from this point on, no dynamic connection)

The created tables become part of the Grist document, defining (e.g., but at this point mainly) dimension tables to be used in Reference columns and as master data. (Furthermore, their content, once populated, will be mirrored to our internal RDB, but we do not need to be concerned about this here, to keep things simple).

The Owner or creator of the document is creating the tables from the template.

For Users of the document it does not matter how the tables were created, and the template may even be removed once the document structure has been finalized.

N.b.: This is all concerning mainly the development process of the Grist Document and does not really concern Viewers or Contributors of the document, once it is published. It is merely thought of as a tool to speed up the development process and foster standardized naming of columns in tables serving a similar purpose.

I will use a different example than yours, that is closer to the real application here. Several dimension tables will share common field names, their content is well defined and identical for all tables.

key | name | label | description | comment | Duplicate_in_key | Duplicate_in_name | Created_at | Created_by | Last_updated_at | Last_updated_by

Bold fields describe the data of the table, to be filled later, italic fields are calculated fields (referencing other fields in this table). The logic of the table is described as in the code block below, taken from Code View.

The Duplicate_in_key column is intended for use with rules lateron to force unique values in the ‘key’ column. (The key will be used as the business key in the RDB outside Grist, just FYI why we want unique values).

This template is intended to build audited dimension tables, where a business key (key) is specified, with a user friendly short name (name), as well as a more verbose description and possibly a comment. Additional columns may be added as desired.

The label column holds the values to be displayed in the drop down when the respective table is used in a reference column, defined to create human readable unique labels derived from the short name and, if needed as a tie breaker, also from the key.

When adding reference columns, we know we always want to select the label column, irrespective of which table we are referring to, as long as it has been created from this template.

@grist.UserTable
class Dim_table_template:
  key = grist.Text()
  name = grist.Text()
  description = grist.Text()
  comment = grist.Text()

  @grist.formulaType(grist.Bool())
  def Duplicate_in_key(rec, table):
    return rec.key != "" and rec.key is not None and len(Dim_table_template.lookupRecords(key=rec.key)) > 1

  @grist.formulaType(grist.Bool())
  def Duplicate_in_name(rec, table):
    return rec.name != "" and rec.name is not None and len(Dim_table_template.lookupRecords(name=rec.name)) > 1

  @grist.formulaType(grist.Text())
  def label(rec, table):
    if rec.Duplicate_in_name:
      return f'{rec.name} ({rec.key})'
    else:
      return rec.name
 
  def _default_Created_at(rec, table, value, user):
    return NOW()
  Created_at = grist.DateTime('Atlantic/Reykjavik')

  def _default_Created_by(rec, table, value, user):
    return user.Name
  Created_by = grist.Text()

  def _default_Last_updated_at(rec, table, value, user):
    return NOW()
  Last_updated_at = grist.DateTime('Atlantic/Reykjavik')

  def _default_Last_updated_by(rec, table, value, user):
    return user.Name
  Last_updated_by = grist.Text()

As it will get a bit wide to specify all columns when we create our tables, I will abbreviate the set of these set of base columns as (BCs) while spelling out any additional columns.

Let’s create a lab inventory system.

Table SDSPhrase: (Safety information):

(BCs)

Table Chemical:

(BCs) | RefList(SDSPhrase) : SDS_Facts | Number : MolWeight | Text : Appearance | Choice : Phase

Table SupplierContact:

(BCs) | Text : Phone

Table Supplier:

(BCs) | Text : Country | Text : Address | RefList(SupplierContact) : Contacts | Choice : Priority

… and so on and so on. There can be tenths of such tables in the end if one designs the hierarchical structure very detailed and granular.

By setting the template up in a smart way, one can avoid “writing” a lot of boilerplate for each of these tables. Say we want to distinguish between Jane Doe in “SuperChem GmbH” and Jane Doe in “ChemSupply Inc.”, using their email as the “key”, it will automatically be added to the “label”, in parentheses after their names. If one would like to rather have the Company Name displayed there, it is one line (or rather one word) to be changed in the formula definition of the label field in the SupplierContact table.

I can describe in more detail or give more examples, but I hope this suffices to get the idea what I am talking about, in a realistic application setting.

All the best and thanks for you time for this conversation, and I am looking forward to hear your thoughts on this.

thanks for these details, very useful. I’m trying to understand if there was a way to avoid the problem altogether and build your tables in a different way, using the relational capabilities of Grist to avoid replicating the schema. But probably you’ve already normalized your database structure and this is what comes out of that exercise. Based on what you shared, I’m at a loss and can’t imagine a way to change your process either…

I think the workaround you’re using (duplicate the table in raw data) is the best approach at the moment. But this is an interesting feature; if there are others who need it, I’m sure the product team will prioritize it.