Key Skills
These are the skills you'll need to put together a Problem Definition.
Last updated
Was this helpful?
These are the skills you'll need to put together a Problem Definition.
Last updated
Was this helpful?
At present, Problem Definitions are defined using . We'll be introducing a graphical way of configuring these in a future update.
When performing queries against a data source - e.g. SQL Server, MongoDB, Elasticsearch - you'll use the query language that's native to that platform.
For example, when using SQL Server, you'd use its flavor of T-SQL. For MySQL, it's a slightly different flavor of T-SQL. With MongoDB, it'd be MQL.
Analyze phase queries are performed using . This allows you to query Parquet files using SQL as if they were tables.
DuckDB is Copyright 2018-2023 Stichting DuckDB Foundation.
For each Parquet file used as in input to the Analyze phase, a kind of virtual table is used. For example, if you have a filename off Foo.parquet then you would have a table named Foo. These virtual tables can then be queried using the .
The end result that DataBug needs to create cases is a Parquet file with "flat" data as opposed to hierarchic data like JSON.
When querying JSON-based data sources, you may be able to flatten your data using the query itself. For example, Cosmos DB's SQL syntax allows you to JOIN between hierarchic layers of a record and SELECT fields from multiple levels.
However, in some cases - especially JSON files or APIs - you can't control the structure of the data you get back from the data source. In these situations, we offer two ways of transforming the data you receive prior to a flat Parquet file being written.
Handlebars is Copyright (C) 2011-2019 by Yehuda Katz
This means that you are responsible for ensuring that you create syntactically-correct JSON as an output. In most cases, JSONata is going to be a far easier way to go.
Cases are created using Markdown and Handlebars.
Use Markdown to present the narrative of a case to your end users. A good structure to follow is:
What's happened;
Why it's important / What the impact is;
How to resolve it, step-by-step.
However, you're free to write whatever works best for your team!
Markdown allows you to control the format and overall wording of your case, but to include data from Parquet files, you'll need to use Handlebars.
Handlebars is Copyright (C) 2011-2019 by Yehuda Katz
Each row from a Parquet file will be presented into your template as a JSON object. Handlebars can then be used to "mail merge" data into your narrative, for example:
Customer {{ CustomerName }}'s contract is close to expiry.
On top of the "out-of-the-box" Handlebars expressions, we have also introduced several of our own to make writing cases easier:
One
Two
Three
is a lightweight querying and transformation language for JSON data. It was inspired by the location path semantics of XPath.
For any given JSON structure, you can transform it into another shape. The gives a good example of what can be achieved and you can use it as a playground for experimentation.
JSONata has a .
is a {{ mustache-based }} templating language. Using Handlebars to transform data gives you complete control over every character of the output JSON you want to create.
The details how to put placeholders from a JSON object into output text. The also lets you experiment with templates.
is a simple and easy-to-use markup language you can use to format virtually any document.
Its allows you to create multiple levels of headings, style fonts, create tables or checklists, etc.
is a {{ mustache-based }} templating language.
The details how to put placeholders from a JSON object into your Markdown content. The also lets you experiment with templates.