Data Architectural Design and Implementation
1) Finance DB | finance-cluster
MySQL Community
2) Tryber DB | prod-crowd[-encrypted-cluster]
Aurora MySQL
3) Unguess DB | unguess-wp
Aurora MySQL
4) ETL | Glue + SNS + EventBridge
There are Data Import and Transformation Glue jobs generated from the two previous DBs that feed Data into the Data Collector. The job list can be found hosted on Glue Ireland.
Each job is scheduled to run at a certain time in the night, so that every job has its own space. The complete scheduling can be found here.
Additionally, EventBridge checks if a job fails and sends an alert to the SNS service, which sends an email with all the details of the failed job to the data@unguess.io mailing list.
5) Data Collector | holistics-postgres
PostgreSQL hosted in Ireland. Documentation about the DB and his Materialized Views (next section) can be found here
6) Data Collector | Materialized Views
We create materialized view inside our Database to have a better and more aggregated view of our data.
7) Holistics
Service used to handle simple and final Data Modelling tasks and, most importantly, [let stakeholders] create Dashboards in a simple way.
8) Redash
Old BI tool. It is connected directly to the raw Data Sources. It is intended for technical users who need to do specific queries (and they know how to do them by themselves).