Prime Video is looking for a Machine Learning Quality Analyst who will have responsibility to assess the output produced by content moderators and machine learning models, assessing the quality and providing vital insights to program, operations, and science teams to improve moderator and machine learning models quality.
The role will require you to be a leader who prioritizes supporting the organization and ensuring defects do not persist. Your role will be to help set and oversee the quality standards of the data the team produces. You will define metrics, identify external signals, and build processes to reduce risk across the programs you support. You will consolidate large data sets into simplified insights that drive business growth and reduce risk.
You will also be responsible for producing reports that show the quality assessments completed by the quality team as well as conducting data analysis to identify key drivers and insights that may be reducing the effectiveness of the models’ outputs. You will produce reports and evidence backed by data and present these findings to science, engineering and program teams.
Key job responsibilities
Leadership
• Knows and communicates Amazon's mission, vision and strategy
• Interfacing with diverse stakeholders such as Program leadership, Operations leads, Technology teams, Business
• Ability to confidently facilitate team discussions and communicate business messages
• Can adapt well to changing circumstances, direction, and strategy
• Demonstrates the ability to give overall direction, performance, coordination and evaluation
• Strong interpersonal and communication skills, good listener, and comfortable interacting with up to Director level
• Ability to organize, prioritize and schedule work assignments
• Experience managing core business KPI’s
Operational Delivery
• Understands operational principles of service delivery and uses data to support continuous improvement
• Gathering requirements and delivering complete data solutions to drive insights and flag risks to Quality, Program, and Operations
• Leading complex analytical deep dives to identify potential process gaps
• Building metrics to analyze key inputs for risk detection and defect reduction
• Recognizing and adopting best practices in reporting and analysis: data integrity, test design, analysis, validation, and documentation
Continuous Improvement
• Uses data to identify areas of ongoing improvement in how content compliance and risk is managed
• Feedback Loop: Continuously communicate with the model development team to provide insights and observations regarding content moderation challenges, and potential improvements for the ML model's performance
• Data Integrity: Ensure the accuracy and integrity of the audited data by performing regular quality checks, identifying and addressing inconsistencies, and maintaining a high standard of data cleanliness
• Adaptation: Keep up-to-date with the latest content policy trends, customer expectations, language requirements, to adapt to audit strategies and guidelines accordingly
A day in the life
This is an individual contributor (IC) role focused on quality assessments and data insights for the program, operations, and quality teams to action and drive improvement.
You will provide reports that include the outcome and quality assessment of the audit and present findings, trends, and insights to business and science teams to improve the models training.
BASIC QUALIFICATIONS
- 1+ years of tax, finance or a related analytical field experience
- 1+ years of business or financial analysis experience
- Experience with Excel
- Experience with SQL
- Experience defining requirements and using data and metrics to draw business insights
- Experience making business recommendations and influencing stakeholders
- Bachelor's degree or equivalent
PREFERRED QUALIFICATIONS
- 5+ years of Excel expertise (including VBA, pivot tables, array functions, power pivots, etc.) and data visualization tools such as Tableau experience
- Demonstrated ability to collaborate with technical and non-technical cross-functional partners
- Ability to influence senior leaders and stakeholders
- Prior experience in a risk organization
- Prior trust & safety risk detection
- Clear and effective verbal and written communication skills
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.