Data Modeling
Last updated
Last updated
The first step of creating a data connector is to design a graph data model for your source data. A graph data model consists of two main components: Nodes and Edges.
To create a graph data model for your source data, you need to identify which entities and relationships are relevant for your use case and how they should be represented as nodes and edges with properties. When modelling your data in a graph, it is useful to think in terms of pairs of edges (such as HasAuthor + AuthorOf, or HasReview + ReviewOf), as relationships are unidirectional in a Curiosity graph.
For example, suppose you want to create a graph database for an online bookstore. You might decide to model your data as follows:
Nodes:
Book: A book entity with properties such as title, genre, price, rating, etc.
Author: A person that has written a book, with properties such as name, website, etc.
Customer: A customer entity with properties such as name, email, address, phone, etc.
Order: An order entity with properties such as date, total amount, status, etc.
Edges:
HasAuthor, AuthorOf: A relationship between books and their authors.
HasOrder, OrderOf: A relationship between a customer and his/her orders.
InOrder, HasItem: A relationship between a book and the order that purchased it.
HasReview, ReviewOf: A relationship between a book and a review.
ReviewBy, WroteReview: A relationship between a review and the customer who wrote it.
You need to define your schema before you can ingest any data. You can do so in two ways: in the user interface, under Manage > Schema, or via the data connector APIs.
You can always change your data types by adding or removing fields, but there are some limitations on what changes you can do. For example, you cannot change the Key field from a data type, nor the data type of a field. If you need to do these changes, you'll need to first delete your data schema, which will also delete all data of the type.
Once you have designed your graph data model for your source data, you need to write code that extracts the relevant information from the source location and creates nodes and edges in your Curiosity Workspace using the Curiosity APIs. Curiosity provides two data-connection libraries, one for writing data connectors in C# and another for connectors in Python code. The includes examples using C# code, but most methods are also available in the Python package.