![]() ![]() EMR Serverless is a serverless option that makes it easy for data analysts and engineers to run Spark-based analytics without configuring, managing, and scaling clusters or servers. ![]() In this section, we show you how to ingest and query data in your transactional data lake in a few steps. On AWS, you can implement a thorough governance strategy with fine-grained access control to the data lake with AWS Lake Formation. One important aspect to a successful data strategy for any organization is data governance. The business zone stores data specific to business cases and applications aggregated and computed from data in the transformed zone. Iceberg provides a table format on top of Amazon S3 in this zone to provide ACID transactions, but also to allow seamless file management and provide time travel and rollback capabilities. The transformed zone is an enterprise-wide zone to host cleaned and transformed data in order to serve multiple teams and use cases. The first zone is the raw zone, where data can be captured from the source as is. AWS provides flexibility and a wide breadth of features to ingest data, build AI and ML applications, and run analytics workloads without having to focus on the undifferentiated heavy lifting.ĭata can be organized into three different zones, as shown in the following figure. In the following figure, we show a comprehensive architecture that uses the modern data architecture strategy on AWS to build a fully featured transactional data lake. Moreover, many customers are looking for an architecture where they can combine the benefits of a data lake and a data warehouse in the same storage location. You can build your modern data architecture with a scalable data lake that integrates seamlessly with an Amazon Redshift powered cloud warehouse. For example, the reason for a flight delay may arrive well after the fact that the fligh is delayed.īuild your transactional data lake on AWS
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |