Data.to.design can ingest data and infer a schema from that to create mappings from. This will work for any source of data from CSV, JSON, APIs.
When we do our inference checks we use the first 10 records, this helps to avoid missing data or ensure all records are consistent. Our inference logic is our best guess at the data type, you can always update what types are via our web app.
Some data types we infer by the key they are called, for example, if the data key is called created_date then we assume that the value will be a DATE.
The most common one and our fallback, if the value is a string of characters we use this type.
i.e. Hello or "My world"
If the raw value is a number or can be parsed as a valid number, then we use this type.
i.e. 3920 or -2043.22
If the raw value is a boolean (true/false) then we use this type.
i.e. true or false
If the data key contains “color” or “colour” in its name.
When we detect valid color names or hex codes then we use this type.
i.e. red or #ff00ff
If the data key contains “date” in its name.