37,000ft Inasight Induction!
So for most of us, our first day at a new company is a bit nondescript… struggling to remember all the names, reading infosec and induction documents, trying to get logged into your new laptop and tiring of constant smiling!
Well our new joiner Dimitrios had good reason to smile as his first day involved picking up his laptop at Heathrow and heading over to Las Vegas for AWS Reinvent.
Here you can see Dimitrios enjoying his Inasight induction at 37,000ft as Neil and I talk him through all the great things we’re doing. So welcome aboard Dimitrios!
Daunting
I’m not sure if I’m the only one out of the 43,000+ others here in Vegas for AWS Reinvent but the sheer size and scale of everything here is overwhelming, if not slightly daunting! The immense number of breakouts, workshops, skill sessions, hands on labs, chalk and talks provides an incredible opportunity for learning. So after finally getting to grips with everything and reaching a level of comfort, on day 1 proper I’ve seen some great reference architectures and examples that back up and reinforce the work we’re doing at Inasight for many of our customers.
Democratisation of data is a theme and driving principle core to much here at Reinvent and of course key for many of our customers. It’s incredible to witness how tech and cloud platforms are transforming the way we store, manage and exploit data. It doesn’t seem that long ago, when investing weeks and months designing data models and complex ETL architectures was the norm. We still need to do these things of course but we’ve now got the flexibility to choose when we invest that time and hopefully that’s when we prove the business value in the data.
The separation of storage and compute, the ability to store structured and semi-structured data in S3, the ability to automatically catalogue this data via AWS Glue and then query via Redshift Spectrum, are great examples of how technology is helping achieve these democratisation goals. Once the value of particular data is proved, then the decision to invest in transforming into more traditional Kimball style dimensional models can begin. You’re then able to benefit from the blazing fast querying that Redshift provides due to it’s MPP architecture and columnar storage.
We’re working with a number of organisations architecting and building strategic BI/ML platforms, exploiting the technologies outlined above, working closely with our customers and re-thinking our approach to ingesting, transforming and exploiting data. It’s been great to see many use cases here at Reinvent, reinforcing the benefits of this approach for organisations.
Glue was announced at last years Reinvent but only in the US and it’s taken a while to hit our shores but I’ve seen it confirmed Glue is available (or will shortly) in Ireland which is something we and our customers have been waiting for (I’ve checked the Regional Product Services table but it’s not yet showing).
On the Redshift side, there’s been some really nice features added, not hugely ground breaking but all good features that enhance the overall product. Query plan caching is nothing new in this space but a really important feature in my view as we see Redshift supporting a wider variety of workloads. This obviously benefits repeatable queries such as dashboards and the like where there’ll be some massive performance benefits. The nice things is it’s all automatic and whilst the performance benefit is great because results are sourced from cache, it’s the fact that it frees the core engine to focus on more demanding workloads that I think is key. In a similar vein, around Work Load Management (WLM), Short Query Acceleration applies ML techniques to classify workloads and those that are deemed ‘short queries’ jump ahead in the queue, so that they don’t sit blocked by long-running queries. Again this is all automatic. There’s also additional monitoring and alerting been added to make the management of Redshift easier.
Oh and if you’re running Redshift today, make sure you upgrade from DC1 to the new DC2 instance type, twice the power for same price, in case you missed the announcement.
What’s Coming?
There were some major announcements in the AI/ML space yesterday including:
We’ve got some special, invite only partner training today on all these new technologies, so I’ll talk through my experience of these in my next post.
Beer in hand also! Welcome!