AWS Ambassador and Inawisdom Head of Practice – Solution Engineering, Phil Basford – talks us through his wish list for the upcoming AWS re:invent which, this year, turns digital….
AWS re:invent 2020 is just over a week away. COVID-19 this year has meant that this annual cloud conference mecca has gone ‘Cloud Native’ by going virtual and is now three weeks long. This means there is no 5-day trip to Las Vegas for me. I will be trying to watch the keynotes in the evening while putting the kids to bed. Maybe it has given AWS a chance to shake things up a little, as once you have been to a few re:invents they can be comfortingly similar. What is guaranteed though is a record-breaking attendance and lots of new services announcements, which is always exciting and highly anticipated so, with that in mind, here are some of my predictions (or is that a wish list) to look out for:
Pay Per Request option for ML Inference
We use SageMaker Endpoints and Batch Transforms a lot for Inawisdom customers to make predictions. However, for clients that require a few predictions infrequently then using both is too much. In this situation we use AWS Lambda, however that has limitations on file sizes (yes you could use EFS, but it just gets complicated) and has no GPU support. Therefore, we would love the ability to run predictions on a managed pay per request SageMaker service.
Fargate GPU support
We have some customers that are Cloud-Agnostic or Multi-Cloud, and in such situations, we cannot use Amazon SageMaker. Therefore, we use Docker and build our own Flask Apps and we deploy these onto ECS Fargate. However, if the ML model requires GPU for training or inference, then ECS Fargate cannot be used as it does not have GPU. Currently, to resolve this, we fire up an EC2-backed instance ECS cluster. We would love for Fargate to get GPU support.
Open Stack Support
Redhat and Amazon announced earlier in the year, their new partnership for Open Stack. For this we even know the service name “Amazon Red Hat OpenShift” and what it is (https://www.redhat.com/en/blog/red-hat-and-aws-extend-collaboration-introducing-amazon-red-hat-openshift). We would just love to see Amazon Red Hat Openshift go GA.
MLFlow and Kubeflow with SageMaker
We would like to see a fully managed service in SageMaker for MLFlow and Kubeflow. It is currently possible to call SageMaker from MLFlow and Kubeflow. However, to run MLFlow and Kubeflow on AWS you need to run a EC2 instance or/and an EKS cluster.
Python support in Code Guru
Code Guru was a 2019 launch, however today it still only supports Java. Our primary language at Inawisdom is Python, as it is the weapon of choice for Data Science. Therefore, we would love Code Guru to support Python, so that we can use it for reviewing code as part of our GIT Pull Requests approval process in Code Commit.
Docker public repo proxy
One sentence that causes us to lose sleep in the middle of the night, other than “National Lockdown” is “ERROR: toomanyrequests: Too Many Requests. You have reached your pull rate limit.”. Docker introduced its rate-limiting this year and in the process, broke lots of CI/CD pipelines that use public images, including AWS SAM. AWS have promised urgently to offer the ability to use ECR to pull public images. It would be great to see this announced.
AWS Glue
We would have predicted lots of AWS Glue announcements; however, the Glue team have been dropping some big new features like DataBrew and Glue Studio for the last two months. Either they beat everyone else to it or something big is coming! We have our fingers crossed there is a big Glue announcement on the way
AWS IoT
Internet of Things applications and services are becoming more prevalent with customers. In a very competitive marketplace, AWS has a comprehensive portfolio. One area perhaps due a new announcement is the Edge computing space. Expect some developments, perhaps a lightweight Greengrass, afocus on running inference at the edge, known as fogging.
Summary
I am going to miss the eating with thousands of people in a car park, I am going to miss cramming into bars. I am going miss being given endless swag (but I do need some socks). I am going to miss the networking from bag drop on the way out to baggage reclaim on the way back. I am going to miss the Strip and the endless parties. I am going to miss getting up at 7am with jet lag and going to the keynotes. I am going to miss working out how to get a DeepLens, DeepRacer or DeepComposer home.
However, AWS have achieved an amazing feat already – shifting this to a virtual conference that looks compelling and packed with amazing content. The sessions look brilliant and I will be watching as many as I can. The keynotes will be fantastic, as ever, and I am really looking forward to the first ever Machine Learning keynote, proving the prevalence and importance of this for customers. I am looking forward to being involved in lots of community events around re:invent including being a pundit for ComSum, which Inawisdom are proudly sponsoring. ComSum will be packed full of content and I am also very excited to be doing a special Machine Learning Fireside Chat with Julien Simon – AWS Global ML Evangelist – and some AWS Heroes that will be shown during the Week in Review session on Friday 11th December at 2pm UK time. Sign up here for the ComSum @re:invent sessions and you’ll be kept abreast of all the highlights and takeaways throughout the whole of re:invent.
If you haven’t signed up for re:invent itself – it’s free this year – and you can register here. I really encourage you to do so – it’s going to be an incredible 3 weeks and you can see how many of my predictions come true!