Rehashing re:Invent 2022

Since its first edition in 2012, re:Invent has been an exciting conference for the computing community around the globe. Not only the actual attendees but millions worldwide are excited to see the launch of new services that will directly or indirectly influence the way we leverage the cloud. AWS has listed all the re:Invent releases here for a quick look. And just like every year, I have a quick recap of some exciting releases. Let’s get started.  

Releases around analytics

There were a lot of announcements around analytics and data processing this year. Amazon security lake – a customer-owned data lake service, Amazon QuickSight Paginated Reports, new QuickSight API capabilities for efficient BI, Amazon Athena for Apache Spark, a new version of AWS Glue, i.e., 4.0, Transcribe, and Amazon OpenSearch Serverless for analytics workloads without managing clusters are some of the standout ones. However, the most interesting one that caught my eye is the additions to Amazon RedShift. 

Amazon RedShift 

With auto-copy from Amazon S3 (available in preview), Redshift can now automatically load files in an Amazon Simple Storage Service (Amazon S3) location that you specify in the data warehouse. You can eliminate the manual copy processing for copy command-enabled data types such as CSV, JSON, Parquet, and Avro. Another upgrade that simplifies data ingestion is Amazon Aurora zero-ETL integration with Amazon Redshift. Multi-AZ deployments and dynamic data masking are other features introduced to enhance security. The general availability of Amazon Redshift integration for Apache Spark and AWS Backup support for Amazon Redshift were also announced at re:Invent this year. General availability of streaming ingestion for Kinesis Data Streams & managed streaming for Apache Kafka and a Spark Connector were some exciting updates for the already strong AWS Redshift. 

Distributed map for AWS Step Functions  

Map state, limited to 40 parallel iterations earlier, created challenges for parallel data processing workloads. In the case of higher workloads, only complex workarounds to the existing map state component were the solution. With the new distributed map state, you can write Step Functions to coordinate much larger-scale parallel workloads within serverless applications than was possible earlier. 

AWS Marketplace updates 

The marketplace is where you can get your hands on readymade solutions around AWS services. However, there is always a concern about the genuineness of the solution or the provider. Not anymore. AWS has introduced Marketplace Vendor Insights. You can access current and validated information for third-party solutions listed on the AWS marketplace. One can also use AWS artifacts to download and assess evidence gathered from the vendors’ security tools and audit reports. 

Releases around Compute  

The releases around compute category had a lot of new stuff, including new AWS SimSpace for spatial simulations, Amazon ECS Service Connect for easy communication between microservices, and a plethora of new EC2 instances. 

To enable additional use cases of AWS’s serverless compute service Lambda SnapStart. With SnapStart, you can reduce the duration of Cold Starts, which can be especially painful for Java environments. After you enable Lambda SnapStart for a particular Lambda function, publishing a new version of the function will trigger an optimization process. The process involves a snapshot of the function just before the invocation begins and storing that snapshot in a multi-tiered cache to make subsequent Lambda container initializations faster. SnapStart is now available for Java 11 functions. 

EC2 updates

AWS Elastic Compute Cloud has been the market leader in the virtual computing instances segment. And just like all the re:Invents before, the 2022 edition has many updates for EC2 users. 

• ENA Express: Enhanced Network Enablers have been a great plus for additional bandwidth. ENA Express uses the Scalable Reliable Datagram (SRD) protocol to reduce the P99 latency of traffic flows by up to 50% and P99.9 latency by up to 85% (in comparison to TCP).

• New EC2 instances: AWS also announced several new EC2 instances that include new general-purpose compute instances - M6in/M6idn, C6in, with higher packet-processing performance and memory-optimized instances R6in/R6idn. The availability of these instances in on-demand & spot form and may be limited to specific regions at launch.   

In addition, AWS announced Inf2 instances (preview) for deep learning (DL) inference applications, C7gn Instances for network-intensive workloads, Hpc7g for tightly coupled compute-intensive HPC and distributed computing workloads, and R7iz Instances for Electronic Design Automation (EDA), financial, actuarial, and simulation workloads. 

Containers 

Direct deployment of third-party applications to Amazon EKS clusters: AWS Marketplace for Containers also added support for direct deployment to Amazon EKS Clusters. In simple words, you can easily find third-party Kubernetes applications on the AWS marketplace from the EKS console itself. It will ease the searching and deploying of third-party Kubernetes apps to EKS clusters.

Database 

If you want to minimize risk and downtime using a blue/green deployment, then AWS has good news. AWS announced fully managed blue/green deployments for Amazon Aurora and Amazon RDS. In addition to this, Amazon RDS has one more exciting update. The relational database, Amazon RDS, now has optimized reads and writes. 

AWS added trusted Language Extensions for PostgreSQL on Amazon Aurora and Amazon RDS. PostgreSQL is slowly becoming a preferred choice of database. More than 85 PostgreSQL extensions are already present in Amazon Aurora and Amazon RDS. Still, if you want to develop your own extensions with the language of your choice, then AWS has you covered. You can safely create extensions for PostgreSQL with the new open-source kit of trusted languages.  

DocumentDB is efficient for JSON workloads. With the announcement of the general availability of Document DB Elastic Clusters, you can now scale your document database to handle virtually any number of writes and reads, with petabytes of storage capacity. Elastic Clusters simplifies how customers interact with Amazon DocumentDB by automatically managing the underlying infrastructure and removing the need to create, remove, upgrade, or scale instances. 

For Developers

Apart from the updates and new releases to the AWS product and service arsenal, AWS also has a few updates for the developers. AWS has introduced AWS Application Composer in preview mode to simplify serverless applications. It will help developers architect and build serverless applications with simple drag, drop, and connect. Another preview that excited us was the Amazon CodeCatalyst. With blueprints, on-demand cloud-based dev environments, issue management, dashboards, unified search, and increased collaboration, this unified software development service will enable active developer collaboration for efficient development and faster delivery.  

More Local Zones your way! 

AWS announced four new local zones in the re:Invent viz., - Buenos Aires, Copenhagen, Helsinki, and Muscat. To know more about leveraging the AWS regions and Zones, refer to our last week’s blog - AWS Regions, Availability Zones, and Strategy & Best Practices.

Releases around Storage

AWS has enhanced its storage capabilities, allowing users to do more with their AWS cloud. Failover controls for Amazon S3 Multi-Region Access Points now enable users to rapidly test and develop highly available applications for business continuity by redirecting S3 data access request traffic flowing through an Amazon S3 Multi-Region Access Point to different AWS Regions.

They also launched the Amazon EFS Elastic Throughput, a new mode for Amazon EFS that lets users use as much throughput as needed and pay only for the throughput that their applications consume. With the help of shared file storage that does not require provisioning or capacity management, this new throughput mode helps users simplify the process of operating workloads and applications on AWS.  

Users can also attach an AWS CloudFormation stack to their data protection policies using AWS Backup. All stateful components supported by AWS Backup are backed up concurrently when users use CloudFormation as a resource. This gives them a single recovery point from which they may restore their application stack or specific resources.

AWS Backup now includes backup support for Amazon Redshift. Applications' data protection can be managed by defining a centralized backup policy using AWS Backup, which can secure Amazon Redshift clusters. In this manner, managing data protection across all supported services is consistent. This service is aimed to enhance user-experience and provide for better data protection management across all supported services.

Failing back Amazon Elastic Compute Cloud (Amazon EC2) instances to the original Region gets more streamlined and faster with the new Automated in-AWS Failback for AWS Elastic Disaster Recovery. It is pretty easy to conduct failover and failback operations (for on-premises or in-AWS recovery) from the AWS Management Console.

Releases around Security, Identity, & Compliance

The AWS KMS External Key Store (XKS), AWS's Key Management Service, lets users with regulatory needs store AWS KMS customer-managed keys on a hardware security module (HSM). Users can run this module on-premises or at any other location they choose.

AWS expanded the Amazon Inspector's reach. It is now capable of scanning and checking for vulnerabilities in AWS Lambda functions too.  

Additionally, AWS launched automated data discovery for Amazon Macie. This new feature helps users see where their private data is stored on Amazon Simple Storage Service (Amazon S3) for a lot less money than it would cost to perform a comprehensive data examination of all S3 buckets.  

Amazon announced a preview of the Amazon Verified Permissions in the regions of the US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Tokyo), and Europe (Ireland). This central framework makes it simple to change, update, and manage fine-grained permissions and permission rules in a single location without changing the code.

Releases around Networking & Content Delivery

AWS Verified Access Preview, a VPN-less Secure Network, lets organizations remotely and securely access corporate applications without needing a VPN. This feature is available in preview in ten AWS Regions: US East (Ohio, N. Virginia), US West (N. California, Oregon), Asia Pacific (Sydney), Canada (Central), Europe (Frankfurt, Ireland, London), and South America (São Paulo).  

Amazon VPC Lattice, available as a preview in the US West (Oregon) Region, offers businesses a standardized means of connecting, securing, and monitoring communication between services. Users may link applications straightforwardly and uniformly across AWS compute services by defining policies for network access, monitoring, and traffic management with VPC Lattice (instances, containers, and serverless functions).  

Releases around Migration & Transfer services

To speed up database migrations, AWS launched a fully managed DMS Schema Conversion, which makes schema assessment and conversion available inside AWS DMS. The AWS Schema Conversion Tool (AWS SCT) can automatically convert the source schema and most of the database code objects, such as functions, views, and stored procedures, to a format compatible with the target engine.  

AWS announced three significant enhancements to the Application Migration Service to ease migration projects of any scale. New Migration Servers Grouping group migration servers into "applications," a collection of servers that work as a single application. The migration stage is managed in "waves." The Updated Launch Template helps users modify the default launch template and other general settings and is used to create the template for launching instances of source servers that will later be installed on Amazon Elastic Compute Cloud (Amazon EC2). The Updated Post-Launch Template allows users to configure custom modernization activities. Any AWS Systems Manager document and its associated parameters can be linked to a post-launch custom action by users.  

Releases around Industries

By storing, querying, and analyzing genomic, transcriptomic, and other omics data, the introduction of Amazon Omics helps researchers, bioinformaticians, and scientists advance scientific knowledge and enhance human health.  

The launch of the AWS Supply Chain, as a preview in the AWS Regions of US East (N. Virginia), US West (Oregon), and Europe (Frankfurt), helps supply chain managers save costs and manage risks more effectively. AWS Supply Chain helps organizations improve customer service levels and cut expenses associated with overstock. It also offers actionable insights powered by machine learning (ML), integrates supply chain data, and offers built-in contextual collaboration.  

Releases around High-Performance Computing

AWS released a brand-new instance type – The Amazon EC2 Hpc6id instance that they designed explicitly for tightly connected HPC applications. As opposed to R6i instances, Amazon EC2 Hpc6id instances have 200 Gbps Elastic Fabric Adapter (EFA) network bandwidth, 1024 GiB RAM, 15.2 TB local SSD storage, and 3.5 GHz 3rd Gen Intel Xeon Scalable CPUs (Ice Lake) that operate at higher rates and optimize high-performance computing.  

Releases around Machine Learning

Amazon's SageMaker has quite a few exciting updates. Apart from a fresh, redesigned UI, SageMaker now supports shadow testing, letting users conduct tests holistically while accounting for current, real-world conditions using the shadow mode. Amazon also revealed the next-gen Amazon SageMaker Notebooks, which claim to boost productivity throughout the ML development process. With the built-in data preparation feature, users can now quickly enhance the quality of their data, collaborate in real-time on the same notebooks with their teams, and instantly turn their notebook code to production ready. The introduction of support for SageMaker Data Wrangler to conduct real-time and batch inference allows users to reuse the data transformation flow and speed up production deployment. Also, since Amazon SageMaker Data Wrangler now supports SaaS applications and data sources, users can use Amazon AppFlow to access data from over 40 SaaS apps and make it available in Amazon SageMaker Data Wrangler. The Amazon SageMaker governance tools are also updated with functionalities that enhance transparency over ML projects, define custom permissions for users, and simplify access control. Another update Amazon SageMaker brought as a preview to the US West (Oregon) Region is the geospatial capability that makes it easy to build, train, and deploy ML models using geospatial data. But this feature is only available as a preview to the US West (Oregon) Region.

The AWS Machine Learning University launched a new educator enablement program to create diverse talent for AI and ML jobs. Apart from this, they upgraded the Amazon Comprehend feature that is used for intelligent document processing (IDP). It can now classify and extract entities from MS Word files, PDF documents, and images without needing to extract the text first. Another enhancement is the Amazon CodeWhisperer for workforce identity solutions. AWS administrators can now use the CodeWhisperer to configure organization-wide settings and efficiently provide access to groups or individuals with Single Sign-On authentication. This is aimed at efficiently cutting down human error while extracting mortgage and loan data. Amazon Textract, a machine learning (ML) service, can perform signature detection, Social Security Number extraction, Tax ID detection, and other data extraction functions to boost the speed and accuracy of mortgage and loan data extraction.

Releases around Management Tools

To determine whether the configuration settings of AWS resources comply with the expected configuration settings, administrators can use the detective mode offered by AWS Config. The AWS Config has been expanded to support the proactive mode so it may be used any time before provisioning and save time by eliminating the need to develop custom pre-deployment validations.  

Building controls that consider service dependencies and relationships takes time and money. AWS launched the  comprehensive controls management  in AWS Control Tower to make it simple for users to apply detective, proactive and managed preventive controls to Organizational Units (OUs) by control objective, compliance framework, and service. The AWS Control Tower does the mapping, saving users time and a lot of manual effort.  

Amazon  CloudWatch Logs  data protection is a new set of features that uses pattern matching and machine learning (ML) to find and safeguard sensitive log data while in transit. Customers from any industry can utilize this functionality and gain native data protection capabilities. Additionally, AWS introduced the Amazon CloudWatch cross-account observability, a new capability that searches, analyzes, and correlates cross-account telemetry data such as traces, logs, and metrics. The CloudWatch cross-account observability provides the flexibility to monitor all the components of applications from a centralized view.

The Amazon CloudWatch Internet Monitor, a feature of CloudWatch that provides visibility into how an internet problem could affect the performance and availability of apps, is a much-needed addition. It helps users diagnose internet problems in a matter of minutes rather than days.

What I think of the releases

As always, the fantastic speaker list rounding out the keynotes always makes re:Invent special. There is a lot to talk about with the plethora of updates and releases. I personally was expecting something special on the container and serverless side. However, the rest of the releases, especially around compute and storage, will compensate and keep me busy for the next few weeks.