Client Background
The client is a well-known company in the travel and leisure industry, boasting more than 1,000 employees. Due to a non-disclosure agreement (NDA), further information cannot be disclosed.
Challenges
PII Exposure & GDPR Compliance
After migrating to a new cloud data warehouse, some sensitive PII was exposed, posing GDPR compliance risks and potential fines, underscoring the need for stronger data privacy measures.
Data Consistency Challenges
The increased number of data producers led to inconsistencies in naming, quality, and modeling practices. The lack of consistent data products risked degrading the system into an unmanageable state.
Complex Access Management
Existing policies for managing access were overly complicated. Users struggled with cumbersome processes, which often resulted in inconsistent provisioning of data access, further complicating compliance and usability.
Scalability Issues
The legacy on-premise database couldn’t support the growing number of users, necessitating a move to an analytical database. This migration surfaced gaps in infrastructure setups and operational processes.
Our Solution
To address these objectives, we took the following steps:
Assessing the current state
We began by evaluating the maturity of data management across different domains. This assessment highlighted areas that needed improvement and provided clarity on where processes were lacking.
Identifying problematic areas and planning improvements
Each domain faced unique challenges—some struggled with data modeling inconsistencies, while others had infrastructure issues or non-standard practices. By identifying these, we prioritized fixes to standardize processes and improve reliability.
Addressing PII data leaks
Inconsistent GDPR compliance led to accidental PII leaks into the cloud. We reviewed the architecture and implemented corrective actions to safeguard sensitive data and prevent breaches.
Prioritizing data products based on criticality
We categorized data products into gold, silver, and bronze tiers to focus development resources on the most critical products, like net revenue tables, ensuring quicker response times and better-quality management.
Automating access management
Access to data products was streamlined using code-based solutions like Terraform, which worked with Okta access groups. This automated system ensured that users could only access the data they were authorized for, significantly reducing manual efforts and the risk of unauthorized access.
At a Glance
Pain Points
- GDPR compliance risks due to PII leaks in cloud data.
- Inconsistencies in data practices across domains, threatening reliability.
- Scalability limitations with legacy on-premise systems.
Key Actions
- Conducted domain maturity assessments and standardized practices.
- Resolved PII exposure by reviewing architecture and applying GDPR-compliant policies.
- Streamlined access management with Terraform and Okta integrations.
- Prioritized critical data products (e.g., gold-level) for focused resource allocation.
Results
- Improved data management maturity across domains.
- Enhanced compliance and streamlined processes.
- Strengthened system scalability and security.
