Modern warehouses rely on complex networks of sensors to enable safe and efficient operations. These sensors must detect everything from packages and containers to robots and vehicles, often in changing environments with varying lighting conditions. More important for Amazon, we need to be able to detect barcodes in an efficient way.

The Amazon Robotics ID (ARID) team focuses on solving this problem. When we first started working on it, we faced a significant bottleneck: optimizing sensor placement required weeks or months of physical prototyping and real-world testing, severely limiting our ability to explore innovative solutions.

To transform this process, we developed Sensor Workbench (SWB), a sensor simulation platform built on NVIDIA’s Isaac Sim that combines parallel processing, physics-based sensor modeling, and high-fidelity 3-D environments. By providing virtual testing environments that mirror real-world conditions with unprecedented accuracy, SWB allows our teams to explore hundreds of configurations in the same amount of time it previously took to test just a few physical setups.

Three key innovations enabled SWB: a specialized parallel-computing architecture that performs simulation tasks across the GPU; a custom CAD-to-OpenUSD (Universal Scene Description) pipeline; and the use of OpenUSD as the ground truth throughout the simulation process.

Parallel-computing architecture

Our parallel-processing pipeline leverages NVIDIA’s Warp library with custom computation kernels to maximize GPU utilization. By maintaining 3-D objects persistently in GPU memory and updating transforms only when objects move, we eliminate redundant data transfers. We also perform computations only when needed — when, for instance, a sensor parameter changes, or something moves. By these means, we achieve real-time performance.

Visualization methods

Sensor Workbench users can pick sphere- or plane-based visualizations, to see how the positions and rotations of individual barcodes affect performance.

This architecture allows us to perform complex calculations for multiple sensors simultaneously, enabling instant feedback in the form of immersive 3-D visuals. Those visuals represent metrics that barcode-detection machine-learning models need to work, as teams adjust sensor positions and parameters in the environment.

CAD to USD

Our second innovation involved developing a custom CAD-to-OpenUSD pipeline that automatically converts detailed warehouse models into optimized 3-D assets. Our CAD-to-USD conversion pipeline replicates the structure and content of models created in the modeling program SolidWorks with a 1:1 mapping. We start by extracting essential data — including world transforms, mesh geometry, material properties, and joint information — from the CAD file. The full assembly-and-part hierarchy is preserved so that the resulting USD stage mirrors the CAD tree structure exactly.

Related content

Two Alexa AI papers present novel methodologies that use vision and language understanding to improve embodied task completion in simulated environments.

To ensure modularity and maintainability, we organize the data into separate USD layers covering mesh, materials, joints, and transforms. This layered approach ensures that the converted USD file faithfully retains the asset structure, geometry, and visual fidelity of the original CAD model, enabling accurate and scalable integration for real-time visualization, simulation, and collaboration.

OpenUSD as ground truth

The third important factor was our novel approach to using OpenUSD as the ground truth throughout the entire simulation process. We developed custom schemas that extend beyond basic 3-D-asset information to include enriched environment descriptions and simulation parameters. Our system continuously records all scene activities — from sensor positions and orientations to object movements and parameter changes — directly into the USD stage in real time. We even maintain user interface elements and their states within USD, enabling us to restore not just the simulation configuration but the complete user interface state as well.

This architecture ensures that when USD initial configurations change, the simulation automatically adapts without requiring modifications to the core software. By maintaining this live synchronization between the simulation state and the USD representation, we create a reliable source of truth that captures the complete state of the simulation environment, allowing users to save and re-create simulation configurations exactly as needed. The interfaces simply reflect the state of the world, creating a flexible and maintainable system that can evolve with our needs.

Application

With SWB, our teams can now rapidly evaluate sensor mounting positions and verify overall concepts in a fraction of the time previously required. More importantly, SWB has become a powerful platform for cross-functional collaboration, allowing engineers, scientists, and operational teams to work together in real time, visualizing and adjusting sensor configurations while immediately seeing the impact of their changes and sharing their results with each other.

New perspectives

In projection mode, an explicit target is not needed. Instead, Sensor Workbench uses the whole environment as a target, projecting rays from the camera to identify locations for barcode placement. Users can also switch between a comprehensive three-quarters view and the perspectives of individual cameras.

Due to the initial success in simulating barcode-reading scenarios, we have expanded SWB’s capabilities to incorporate high-fidelity lighting simulations. This allows teams to iterate on new baffle and light designs, further optimizing the conditions for reliable barcode detection, while ensuring that lighting conditions are safe for human eyes, too. Teams can now explore various lighting conditions, target positions, and sensor configurations simultaneously, gleaning insights that would take months to accumulate through traditional testing methods.

Related content

Amazon researchers draw inspiration from finite-volume methods and adapt neural operators to enforce conservation laws and boundary conditions in deep-learning models of physical systems.

Looking ahead, we are working on several exciting enhancements to the system. Our current focus is on integrating more-advanced sensor simulations that combine analytical models with real-world measurement feedback from the ARID team, further increasing the system’s accuracy and practical utility. We are also exploring the use of AI to suggest optimal sensor placements for new station designs, which could potentially identify novel configurations that users of the tool might not consider.

Additionally, we are looking to expand the system to serve as a comprehensive synthetic-data generation platform. This will go beyond just simulating barcode-detection scenarios, providing a full digital environment for testing sensors and algorithms. This capability will let teams validate and train their systems using diverse, automatically generated datasets that capture the full range of conditions they might encounter in real-world operations.

By combining advanced scientific computing with practical industrial applications, SWB represents a significant step forward in warehouse automation development. The platform demonstrates how sophisticated simulation tools can dramatically accelerate innovation in complex industrial systems. As we continue to enhance the system with new capabilities, we are excited about its potential to further transform and set new standards for warehouse automation.

When Andy Jassy, then head of Amazon Web Services, announced Amazon Aurora in 2014, the pitch was bold but metered: Aurora would be a relational database built for the cloud. As such, it would provide access to cost-effective, fast, and scalable computing infrastructure.

In essence, he explained, Aurora would combine the cost effectiveness and simplicity of MySQL with the speed and availability of high-end commercial databases, the kind that firms typically managed on their own. In numbers, Aurora promised five times the throughput (e.g., the number of transactions, queries, read/write operations) of MySQL at one-tenth the price of commercial database solutions, all while offloading costly management challenges and maintaining performance and availability.

Aurora launched a year later, in 2015. Significantly, it decoupled computation from storage, a distinct contrast to traditional database architectures where the two are entwined. This fundamental innovation, along with automated backups and replication and other improvements, enabled easy scaling for both computational tasks and storage, while meeting reliability demands.

“Aurora’s design preserves the core transactional consistency strengths of relational databases. It innovates at the storage layer to create a database built for the cloud that can support modern workloads without sacrificing performance,” explained Werner Vogels, Amazon’s CTO, in 2019.

“To start addressing the limitations of relational databases, we reconceptualized the stack by decomposing the system into its fundamental building blocks,” Vogels said. “We recognized that the caching and logging layers were ripe for innovation. We could move these layers into a purpose-built, scale-out, self-healing, multitenant, database-optimized storage service. When we began building the distributed storage system, Amazon Aurora was born.”

Within two years, Aurora became the fastest-growing service in AWS history. Tens of thousands of customers — including financial-services companies, gaming companies, healthcare providers, educational institutions, and startups — turned to Aurora to help carry their workloads.

In the intervening years, Aurora has continued to evolve to suit the needs of a changing digital landscape. Most recently, in 2024, Amazon announced Aurora DSQL. A major step forward, Aurora DSQL is a serverless approach designed for global scale and enhanced adaptability to variable workloads.

Today, International Data Corporation (IDC) research estimates that firms using Aurora see a three-year return on investment of 434 percent and an operational cost reduction of 42 percent compared to other database solutions.

But what lies behind those figures? How did Aurora become so valuable to its users? To understand that, it’s useful to consider what came before.

A time for reinvention

In 2015, as cloud computing was gaining popularity, legacy firms began migrating workloads away from on-premises data centers to save money on capital investments and in-house maintenance. At the same time, mobile and web app startups were calling for remote, highly reliable databases that could scale in an instant. The theme was clear: computing and storage needed to be elastic and reliable. The reality was that, at the time, most databases simply hadn’t adapted to those needs.

Amazon engineers recognized that the cloud could enable virtually unlimited, networked storage and, separately, compute.

That rigidity makes sense considering the origin of databases and the problems they were invented to solve. The 1960s saw one of their earliest uses: NASA engineers had to navigate a complex list of parts, components, and systems as they built spacecraft for moon exploration. That need inspired the creation of the Information Management System, or IMS, a hierarchically structured solution that allowed engineers to more easily locate relevant information, such as the sizes or compatibilities of various parts and components. While IMS was a boon at the time, it was also limited. Finding parts meant engineers had to write batches of specially coded queries that would then move through a tree-like data structure, a relatively slow and specialized process.

In 1970, the idea of relational databases made its public debut when E. F. Codd coined the term. Relational databases organized data according to how it was related: customers and their purchases, for instance, or students in a class. Relational databases meant faster search, since data was stored in structured tables, and queries didn’t require special coding knowledge. With programming languages like SQL, relational databases became a dominant model for storing and retrieving structured data.

By the 1990s, however, that approach began to show its limits. Firms that needed more computing capabilities typically had to buy and physically install more on-premises servers. They also needed specialists to manage new capabilities, such as the influx of transactional workloads — as, for instance, when increasing numbers of customers added more and more pet supplies to virtual shopping carts. By the time AWS arrived in 2006, these legacy databases were the most brittle, least elastic component of a company’s IT stack.

The emergence of cloud computing promised a better way forward with more flexibility and remotely managed solutions. Amazon engineers recognized that the cloud could enable virtually unlimited, networked storage and, separately, computation.

The Amazon Relational Database Service (Amazon RDS) debuted in 2009 to help customers set up, operate, and scale a MySQL database in the cloud. And while that service expanded to include Oracle, SQL Server, and PostgreSQL, as Jeff Barr noted in a 2014 blog post, those database engines “were designed to function in a constrained and somewhat simplistic hardware environment.”

AWS researchers challenged themselves to examine those constraints and “quickly realized that they had a unique opportunity to create an efficient, integrated design that encompassed the storage, network, compute, system software, and database software”.

“The central constraint in high-throughput data processing has moved from compute and storage to the network,” wrote the authors of a SIGMOD 2017 paper describing Aurora’s architecture. Aurora researchers addressed that constraint via “a novel, service-oriented architecture”, one that offered significant advantages over traditional approaches. These included “building storage as an independent fault-tolerant and self-healing service across multiple data centers … protecting databases from performance variance and transient or permanent failures at either the networking or storage tiers.”’

The serverless era is now

In the years since its debut, Amazon engineers and researchers have ensured Aurora has kept pace with customer needs. In 2018, Aurora Serverless provided an on-demand autoscaling configuration that allowed customers to adjust computational capacity up and down based on their needs. Later versions further optimized that process by automatically scaling based on customer needs. That approach relieves the customer of the need to explicitly manage database capacity; customers need to specify only minimum and maximum levels.

Achieving that sort of “resource elasticity at high levels of efficiency” meant Aurora Serverless had to address several challenges, wrote the authors of a VLDB 2024 paper. “These included policy issues such as how to define ‘heat’ (i.e., resource usage features on which to base decision making)” and how to determine whether remedial action may be required. Aurora Serverless meets those challenges, the authors noted, by adapting and modifying “well-established ideas related to resource oversubscription; reactive control informed by recent measurements; distributed and hierarchical decision making; and innovations in the DB engine, OS, and hypervisor for efficiency.”

As of May 2025, all of Aurora’s offerings are now serverless. Customers no longer need to choose a specific server type or size or worry about the underlying hardware or operating system, patching, or backups; all that is completely managed by AWS. “One of the things that we’ve tried to design from the beginning is a database where you don’t have to worry about the internals,” Marc Brooker, AWS vice president and Distinguished Engineer, said at AWS re:Invent in 2024.

These are exactly the capabilities that Arizona State University needs, says John Rome, deputy chief information officer at ASU. Each fall, the university’s data needs explode when classes for its more than 73,000 students are in session across multiple campuses. Aurora lets ASU pay for the computation and storage it uses and helps it to adapt on the fly.

We see Amazon Aurora Serverless as a next step in our cloud maturity.

John Rome, deputy chief information officer at ASU

“We see Amazon Aurora Serverless as a next step in our cloud maturity,” Rome says, “to help us improve development agility while reducing costs on infrequently used systems, to further optimize our overall infrastructure operations.”

And what might the next step in maturity look like for the now 10-year-old Aurora service? The authors of that 2024 paper outlined several potential paths. Those include “introducing predictive techniques for live migration”; “exploiting statistical multiplexing opportunities stemming from complementary resource needs”, and “using sophisticated ML/RL-based techniques for workload prediction and decision making.”

• We’re exploring Meta’s Federation Platform, a scalable set of tools for managing compliance-related tasks, along with Privacy Waves, our method for batching these tasks and ensuring accountability. 
• Together, the Federation Platform and Privacy Waves create a structured, effective, and sustainable approach to operationalizing privacy work, enabling Meta to safeguard user data for the billions of people that use our products.
• Given its success in the privacy domain, we’re expanding this approach to other domains such as security and accessibility.

At Meta, we take a systematic approach to privacy-related compliance. Experts decode complex obligations into actionable product requirements, ensuring coverage and consistency across all Meta products. We then deploy technical solutions that address these requirements at scale through our Privacy Aware Infrastructure (PAI) initiative. Following that, our privacy teams centrally automate remediation of potential issues; and finally, if expert help is needed, they send tasks to product teams for distributed execution.

Operationalizing this work at Meta’s scale – across tens of thousands of engineers and numerous products – requires robust coordination. To facilitate this, we developed the Federation Platform and Privacy Waves program:

• The Federation Platform breaks down large compliance-related initiatives into smaller, manageable workstreams. It distributes tasks to the appropriate teams and enables them to track progress through to completion.
• The Privacy Waves program organizes tasks for these initiatives into monthly batches, creating a predictable cadence that improves quality and accountability of task distribution and management. It helps teams plan and execute their compliance-related work systematically, rather than reactively. 

Together, the Federation Platform and Privacy Waves program play a critical role in safeguarding user data and ensuring consistent, effective operations of our systems and solutions, supporting Meta’s compliance posture (for both existing and future obligations) while balancing internal engineering efficiency and experience. 

They are significant levers in Meta’s compliance-related efforts, managing over 100,000 tasks annually within established timelines. Internal surveys reveal significantly higher positive sentiment for Privacy Waves tasks compared to ad-hoc tasks. And we estimate that the program has saved hundreds of thousands of engineering hours by enhancing strategy, tooling, and task quality. The success of this approach in the privacy domain has encouraged its expansion into other domains such as security, accessibility and our broader compliance efforts.

The need for a centralized work distribution and management system

There are several reasons why large organizations like Meta benefit from a centralized system to distribute and manage compliance-related work:

• Meeting privacy obligations at scale is complex because it often requires thousands of engineers to each complete small, specialized tasks across hundreds of global pressures and thematic areas.

• Scalability and internal accountability are crucial. Doing this ad hoc can lead to task fatigue, difficulty meeting completion expectations, and diminished developer sentiment. Without centralized management and oversight, it becomes challenging to effectively prioritize, track, and execute work across organizational boundaries, or to deduplicate tasks across teams.

• Developer experience matters and can even increase output. A positive, well-managed task flow reduces operational burden, maintains morale, and sustains high productivity. 

• External accountability is essential to operations. Meta must demonstrate consistent and effective operations to regulators and auditors. The Federation Platform enables clear, standardized practices along with consistent documentation and validation to uphold Meta’s compliance posture in response to external requirements.

Managing privacy work with the Federation Platform

Workstream configuration: How engineers integrate with the platform

Implementing a workstream on the Federation Platform requires defining in-code logic that mirrors the typical lifecycle of a potential privacy issue. This involves specifying how to detect, distribute, remediate, and verify resolution of these issues, ensuring their effective management. The resulting technical workstream configuration (code file) includes methods for:

• Scraping flags: Scraping involves identifying the relevant set of privacy flags –  indications of potential issues that require attention. These flags are ingested into the Federation Platform based on the workstream’s configuration, which often leverages Meta’s reusable detection and verification frameworks. The scraping process can be automated to run daily using in-code methods or ad-hoc via the platform’s intake APIs. Scraping defines the scope of the workstream, with additional filters and linters configured as needed.
• Ownership resolution: This involves implementing logic to determine the ownership of privacy flags. Typically, this requires referencing Meta’s central catalog to map relevant assets, such as code files and data tables, to their respective owners.

• Grouping: Workstreams can optionally group related flags, such as those with a common owner or located in the same directory. This allows for efficient bulk remediation by bundling these flags into a single task or diff (code change).

• Actioning (Task/Diff): Workstreams decide how to address each privacy flag or group of flags. The most common approach is to file a task, which is then assigned to the asset owner. Alternatively, they can choose to send automated code changes to directly resolve issues, which must be reviewed by the asset owner.

• Task content and distribution: Workstreams configure the content of tasks, providing context on why the task is necessary, its alignment with privacy initiatives, and instructions and workflows to fix the issue. Workstreams also configure how they want to distribute their tasks, which is most commonly done through the Privacy Waves program.

• Resolution logic: Finally, workstreams define resolution logic to determine when a privacy flag is resolved. This allows the Federation Platform to automatically close tasks once the underlying issue is fixed or reopen tasks if they are prematurely closed.

The general-purpose configuration described above is versatile and extends well beyond privacy use cases. For instance, security and accessibility workstreams have started utilizing it to address potential vulnerabilities and product accessibility matters through task distribution. Similarly, engineering excellence initiatives operate workstreams to drive API migrations, code quality improvements, and the cleanup of obsolete experiments across numerous teams. This positions the Federation Platform as a powerful tool for driving diverse, large scale initiatives across the organization.

In addition to the technical configuration steps outlined above, privacy workstreams strive to adhere to the comprehensive end-to-end federation process detailed below, ensuring a holistic approach to managing privacy issues.

An overview of the end-to-end federation process

Step 1: High-level strategy and planning

Before distributing work, a thorough review process evaluates the holistic strategy for a privacy area to ensure their plan efficiently meets applicable privacy-related obligations. This strategy often involves a combination of developing privacy aware infrastructure and controls through traditional project work, privacy teams centralizing bulk remediation via scripts and mass code changes, and – when automated solutions are not feasible – distributing work across the company via Federation Platform workstreams and Privacy Waves.

Product organizations (e.g., Facebook, Instagram, WhatsApp) receive advanced visibility into upcoming privacy work, allowing them to incorporate it into their roadmaps and commit to its delivery. While aligning work across organizational lines takes longer, it ultimately enables easier and more efficient completion of tasks.

Step 2: Configuring efficient task experiences

Tasks for Federation Platform workstreams that participate in Privacy Waves must clearly communicate the nature of the work, due dates, link to relevant context and documentation, and contain the necessary steps for resolution. Structured tasks guide users through a wizard-like workflow with multiple-choice questions, often culminating in automated remediations (e.g., code changes, click-to-fix tools) based on user decisions. These ‘wizards’ facilitate appropriate decision-making by product engineers and, in some cases, have been shown to reduce the effort required to complete tasks by around 50%.

Tasks are enriched with links to support forums and similar tasks where assistance can be sought, if needed. AI-powered support agents are embedded within tasks which help task owners search through relevant resources and write code quickly, which requires human review before landing.

Step 3: Reviewing and improving task quality

A review committee provides feedback on task quality and content for workstreams participating in Privacy Waves, identifying areas for improvement and opportunities for automation. Automated health signals for each workstream, such as completion rates, open tasks, deferral rates, and developer friction (e.g., broken tooling, inadequate support), are measured and tracked. Workstreams and their reviewers monitor these metrics monthly and are held accountable for improvements.

Engineering sentiment is captured for each workstream through task owner surveys, and AI is used to summarize their feedback, enabling workstream owners to learn from task owner input and enhance future tasks. These features contribute to improved work quality, developer sentiment, and completion rates.

Step 4: Distributing the work

Linting tools are employed to prevent the distribution of low-quality and low-risk work (e.g., for assets queued for deletion or lacking any data). Workstreams can configure the lints they wish to apply.

Tasks are sent in Privacy Waves, which are batches of privacy-related work distributed at a predefined, predictable cadence. Privacy Waves streamline execution, coordination, and reporting, since all tasks in a wave share the same deadline, allowing for timely reminders.

A sophisticated matching algorithm aligns tasks with teams based on competing priorities related to assets they own. Combined with predictable task distribution, this approach ensures timely work assignment and enables teams to effectively prioritize, allowing them to balance responsibilities and make consistent progress towards addressing their workloads.

Step 5: Ensuring accountability of execution

To ensure timely completion of tasks, deadlines are established with an aim at preventing deferral beyond these critical dates. Automated nudges and escalations are strategically used to remind individuals and teams to complete work on schedule, minimizing unnecessary noise and highlighting overdue tasks that require immediate attention.

Furthermore, completion rates for privacy work are rigorously measured and reported at all organizational levels, fostering a culture of accountability from frontline teams to leadership. This transparent approach ensures that everyone is held responsible for executing their tasks in a timely manner, promoting a sense of ownership and urgency across the organization.

Step 6: Reporting and recognition

The centralized distribution of tasks via Federation Platform and Privacy Waves streamline operational effectiveness and verification. These systems document completed tasks in a standardized format that aligns with expectations, providing clear and consistent evidence that supports Meta’s compliance posture in response to external requirements.

At Meta, executing on compliance-related work is an integral part of internal engineering expectations. To ensure that individuals receive the recognition they deserve, centralized recognition tooling is utilized to credit their contributions in performance evaluations. This approach not only motivates engineers to prioritize these efforts, but also reinforces the importance of this critical work in maintaining user trust and our compliance posture.

Expansions for the Federation Platform and Waves

As Meta continues to evolve, the Federation Platform and Waves programs are actively being expanded into new domains like security, accessibility, and broader compliance-related efforts. This expansion presents unique challenges, including different types of tasks, complex multi-step remediation processes, varying deadlines, and more. However, our foundational principles of centralized task distribution, execution tracking, and accountability provide a robust framework to address these challenges effectively.

To ensure seamless extension into new areas, we’ll refine our tooling and processes, developing solutions that cater to each domain’s specific needs while maintaining high standards of quality and efficiency. By doing so, we aim to exceed expectations, reinforcing our commitment to safeguarding user data and ensuring efficient and consistent operations across all areas. This forward-looking approach underscores Meta’s dedication to innovation in compliance standardization, setting a benchmark for other tech companies to follow.

Acknowledgments

The authors would like to express our gratitude to reviewers of this post, including (in last name alphabetical order): Chris Adams, Bob Baldwin, Denys Besedynskyy, Herb David, Dylan Drop, Katriel Cohn-Gordon, Xenia Habekoss, Mohit Jha, Ryan Pratt, Matt Pregozen, Jessica Retka, Thomas Richards, and Chris Wiltz, many of whom have made significant contributions to Federation Platform and Privacy Waves.

Additionally, we would also like to acknowledge the contributions of many current and former Meta employees, who have played a crucial role in developing and maturing Federation Platform and Privacy Waves over the years. In particular, we would like to extend special thanks to (in last name alphabetical order): Quinn Armstrong, Cecilia Baek, Yashdeep Bindal, Chris Buckley, Adam Campbell, Katriel Cohn-Gordon, Ruo Ding, Jason Fennell, Andrew Fong, Riccardo Govoni, Abhishek Gulati, Aleksandar Ilic, AJ Jahansouz, Shruthi Katakam, Risa Kawai, [Emile Litvak], Amira Malpass, Idan Michael, Jason Nawrocki, Anthony O’Sullivan, Yuval Oren, Disha Parekh, [Uday Patireddy], Vimalkumar Patel, [Riley Pinkerton], Matt Pregozen, Mateen Saifyan, Pallavi Saraswati, Jay Shah, Or Sperling, Sana Surani, Rajesh Vantipalli, Avi Varadarajulu, Michelle Xu, Robbin Xu, Rui Xue, Anna Zeng, and Hansen Zhang.