As a Data Engineer, the ideal candidate will partner with Software Developers, Business Intelligence Engineers, Scientists, and Product Managers to develop scalable and maintainable data pipelines for both structured and unstructured (text-based) data. This candidate should have strong business judgment, a good sense of architectural design, excellent written/documentation skills, and experience with big data technologies (Spark/Hive, Redshift, EMR, and other AWS technologies). This role involves both overseeing existing data infrastructure and data pipelines, as well as developing new ones.
Key job responsibilities
- Be hands-on with ETL to build data pipelines to support automated reporting.
- Interface with other technology teams to extract, transform, and load data from a wide variety of data sources.
- Implement data structures using best practices in data modeling, ETL/ELT processes, and SQL, Redshift.
- Model data and metadata for ad-hoc and pre-built reporting.
- Build robust and scalable data integration (ETL) pipelines using SQL, Python and Spark.
- Build and deliver high quality data sets to support business intelligence engineers, data scientists, economists, and customer reporting needs.
- Manage AWS resources including Glue, Redshift, MWAA, EMR, Lambda.
- Diagnose and resolve operational issues, perform detailed root cause analysis, respond to suggestions for enhancements.
A day in the life
As a Data Engineer, you will be working with cross-functional partners from Science, Product, SDEs, Operations and leadership to help them translate raw data into actionable insights for stakeholders by building data pipelines, choosing the right data model, empowering them to make data-driven decisions.
About the team
Customer Experience and Business Trends is an organization made up of a diverse suite of functions dedicated to deeply understanding and improving customer experience, globally. We are a team of builders that develop products, services, ideas, and various ways of leveraging data to influence product and service offerings - for almost every business at Amazon - for every customer (e.g., consumers, developers, sellers/brands, employees, investors, streamers, gamers).
Our approach is based on determining the customer need, along with problem solving, and we work backwards from there. We use technical and non-technical approaches and stay aware of industry and business trends. We are a global team, made up of a diverse set of profiles, skills, and backgrounds – including: Product Managers, Software Developers, Computer Vision experts, Solutions Architects, Data Scientists, Business Intelligence Engineers, Business Analysts, Risk Managers, and more.
BASIC QUALIFICATIONS
- 1+ years of data engineering experience
- Experience with data modeling, warehousing and building ETL pipelines
- Experience with one or more query language (e.g., SQL, PL/SQL, DDL, MDX, HiveQL, SparkSQL, Scala)
- Experience with one or more scripting language (e.g., Python, KornShell)
- Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
PREFERRED QUALIFICATIONS
- Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
- Experience with data visualization software (e.g., AWS QuickSight or Tableau) or open-source project
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.