Wednesday, August 30, 2023

Economics & Optimization of AI/ML

Luis Ceze (@luisceze, Founder/CEO @OctoML) talks about barriers to entry for AI & ML, the economics of funding, training, fine tuning, inferencing and optimizations.

SHOW: 749

CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw

NEW TO CLOUD? CHECK OUT - "CLOUDCAST BASICS"

SHOW SPONSORS:

  • CloudZero – Cloud Cost Visibility and Savings
  • ​​CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
  • Reduce the complexities of protecting your workloads and applications in a multi-cloud environment. Panoptica provides comprehensive cloud workload protection integrated with API security to protect the entire application lifecycle.  Learn more about Panoptica at panoptica.app

SHOW NOTES:

Topic 1 - Welcome to the show. You have an interesting background with roots in both VC markets and academia. Tell us a little bit about your background.

Topic 2 - Generative AI is now all the rage. But as more people dig into AI/ML in general, they find out quickly there are a few barriers to entry. Let’s address some of them as you have an extensive history here. The first barrier I believe most people hit is complexity. The tools to ingest data into models and deployment of models has improved but what about the challenges implementing that into production applications? How do folks overcome this first hurdle?

Topic 3 - The next hurdle I think most organizations hit is where to place the models. Where to train them, where to fine tune them and where to run them could be the same or different places. Can you talk a bit about placement of models? Also, as a follow up, how does GPU shortages play into this and can models be fine tuned to work around this?

Topic 4 - Do you see the AI/ML dependence on GPU’s continuing into the future? Will there be an abstraction layer or another technology coming that will allow the industry to move away from GPU’s from more mainstream applications?

Topic 5 - The next barrier but very related to the previous one is cost. There are some very real world tradeoffs between cost and performance when it comes to AI/ML. What cost factors need to be considered besides hardware costs? Data ingestion and data gravity comes to mind as a hidden cost that can add up quickly if not properly considered. Another one is latency. Maybe you arrive at an answer but at a slower rate that is more economical. How do organizations optimize for cost?

Topic 6 - Do most organizations tend to use an “off the shelf model” today? Maybe an open source model that they train with their private data? I would expect this to be the fastest way to production, why build your own model when the difference is in your data? How does data privacy factor into this scenario?

FEEDBACK?

Posted by

Sunday, August 27, 2023

Considerations for Enterprise AI

Let’s talk through some of the challenges that Enterprises will have with AI - from data location to GPU location, to model biases, to data privacy to training vs. execution.

SHOW: 748

CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw

CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"

SHOW SPONSORS:

SHOW NOTES:


ARE THERE EXPECTATIONS OF “OLD AI” vs. “NEW AI”?

  • Are business leaders thinking about unique AI applications and use-cases, or just “ChatGPT-everything”?
  • Formal data scientists vs. citizen data scientists?
  • Will this just be an application, or have an impact on every aspect of a business and the IT industry?

WILL ENTERPRISE AI BE DIFFERENT THAN CONSUMER AI? 

  • The industry is actively working on a broad set of models that can be used for different use-cases. 
  • It's commonly accepted that AI models need to be trained near the sources of data. 
  • Many businesses are concerned about including their company data into these public models
  • Many businesses will want to deploy tuned models and applications in data center, public cloud and edge environments. 
  • New AI applications will be required to meet security, regulatory and compliance standards, like other business applications. 


FEEDBACK?

Posted by

Wednesday, August 23, 2023

Streaming alternatives to Kafka

Yaniv Ben Hemo (@yanivbh1, Founder/CEO at @memphis_Dev) talks about Memphis Cloud, an alternative architecture to delivering streaming data for applications.  

SHOW: 747

CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw

NEW TO CLOUD? CHECK OUT - "CLOUDCAST BASICS"

SHOW SPONSORS:

SHOW NOTES:


Topic 1 - Welcome to the show. Tell us a little bit about your background, and what brought you to create Memphis.Dev

Topic 2 - Let’s start at the beginning. Most folks will want to know why a streaming alternative. Isn’t Kafka good enough? What challenges did you personally encounter?

Topic 3 - In reviewing the architecture, it mentions differences between a broker and a streaming stack. Can you elaborate on what that means? What components are typically needed for a proper data streaming solution?

Topic 4 - One of the common issues with Kafka I hear about is operations complexity over time. It isn’t uncommon that the more a system scales, the more complex it is to operate and also maybe the harder it is to get insights and mine for key data for instance. Have you seen this in your experience?

Topic 5 - Let’s talk use cases. How do you envision organizations using Memphis Cloud? What problems are you trying to solve in the market? Is Memphis Cloud a SaaS offering? How would it be implemented in an organization?

Topic 6 - The data management side of all of this to always be problematic. Where and how is the data managed? What does the lifecycle of the data look like and what design considerations went into this aspect?

Topic 7 - When building large distributed streaming systems, I’m sure there are trade offs and optimizations of features to consider. What are you optimizing for and what are the design tradeoffs developers need to consider?


FEEDBACK?

Posted by
Labels: , , , ,

Sunday, August 20, 2023

What is AWS after crossing the Chasm?

As the competitive cloud landscape is shifting, let’s take a look at some possibilities of what AWS might look like after they cross the chasm. 

SHOW: 746

CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw

CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"

SHOW SPONSORS:

SHOW NOTES:


IF ONLY 10-15% OF APPS ARE IN THE CLOUD, HAVE WE CROSSED THE CHASM?

  • AWS is $85B/yr business, after 17 years
  • AWS claims that 10-15% of IT is in the cloud
  • AWS has attracted startups, and mostly competes against legacy IT companies

AWS MOATS AND WHAT MIGHT COME NEXT?

  • Amazon/AWS has always made large CAPEX investments
  • AWS claims to have the largest farm of GPUs, and ARM servers
  • Open source projects are moving to licensing that reduces competition from AWS
  • AWS growth rate has been slowing since Q4 2021
  • Innovation? Application Portfolio? Pricing vs. Profitability?
  • AWS has done limited acquisitions and partners are kept at arms-length (vs. OpenAI / MSFT)
  • AWS seems to be behind in the AI race, although still very early in the market maturity
  • AWS doesn’t have a large set of “owned/branded” applications
  • What does a future AWS look like that is mostly infrastructure? 


FEEDBACK?

Posted by

Wednesday, August 16, 2023

Understanding Machine Learning Features and Platforms

Gaetan Castelein (@gaetcast, VP Marketing at @tectonai) talks about the complexities of building AI models, features and deploying AI into production for real-time applications. 


SHOW: 745

CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw

NEW TO CLOUD? CHECK OUT - "CLOUDCAST BASICS"

SHOW SPONSORS:

  • AWS Insiders is an edgy, entertaining podcast about the services and future of cloud computing at AWS. Listen to AWS Insiders in your favorite podcast player. 
  • Cloudfix Homepage
  • Find "Breaking Analysis Podcast with Dave Vellante" on Apple, Google and Spotify
  • Keep up to data with Enterprise Tech with theCUBE
  • Reduce the complexities of protecting your workloads and applications in a multi-cloud environment. Panoptica provides comprehensive cloud workload protection integrated with API security to protect the entire application lifecycle.  Learn more about Panoptica at panoptica.app

SHOW NOTES:

Topic 1 - Welcome to the show. Tell us a little bit about your background

Topic 2 - Let’s start with some terminology. A lot of our listeners might be relatively new to Machine Learning. I’m still coming up to speed and I actually spent more time than usual just wrapping my head around the concepts and terms and piecing them all together. What is a feature? Why is it important? How many features does ChatGPT 3 have or ChatGPT4?

Topic 3 - How is a feature different from a model? Both are needed, why?

Topic 4 - I’ve always wondered exactly what a data scientist does. Is this where the term Feature Engineering comes into play? Who turns the data into features and picks the appropriate model? 

Topic 5 - Early Machine Learning was analytical ML (offline/batch), correct? How is that different from operational ML (online/batch) and real-time ML?

Topic 6 - Now that we have all that out of the way. What is a Feature Platform? How does it integrate into an organization’s existing Devops workflows and/or CI/CD pipelines? (Features as Code) How is it different from a Feature Store?

Topic 7 - How do you know if the features + model yield a good result? How is prediction accuracy typically measured?

FEEDBACK?

Posted by
Labels: , , , , ,

Sunday, August 13, 2023

Cloud and Software Economics

If every company has become a software company, is it still easy to be a profitable software company? Lots of things are changing around the economics of software.

SHOW: 744

CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw

CHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"

SHOW SPONSORS:

SHOW NOTES:

BEING A SOFTWARE COMPANY - JUST BUILD A MOAT

  • Are we sure every company wants to become a software company?
  • Lots of software licensing changes in the news
  • Making money as a software business, at scale, is difficult

PEOPLE DON’T LIKE MOATS AROUND THEIR SOFTWARE

  • Open source is good for collaboration
  • Open source could be good for marketing
  • Software margins attract VCs
  • Open source mostly hates software margins
  • What happens to all the software companies funded by VCs over the last 3 yrs?

FEEDBACK?

Posted by

Wednesday, August 9, 2023

Trends in API Security

Filip Verloy (@filipv, Field CTO at @NonameSecurity) talks about the latest trends in API security, how you could be a victim of a Moveit attack, and more

SHOW: 743

CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw

NEW TO CLOUD? CHECK OUT - "CLOUDCAST BASICS"

SHOW SPONSORS:

  • Reduce the complexities of protecting your workloads and applications in a multi-cloud environment. Panoptica provides comprehensive cloud workload protection integrated with API security to protect the entire application lifecycle.  Learn more about Panoptica at panoptica.app
  • Find "Breaking Analysis Podcast with Dave Vellante" on Apple, Google and Spotify
  • Keep up to data with Enterprise Tech with theCUBE

SHOW NOTES:

Topic 1 - Welcome to the show. We’ve worked together in the past at previous companies, it’s great to catch up again. For those out there that don’t know you, tell us a little bit about your background, and how you got involved in API security.

Topic 2 - We keep hearing about APIs and API security but in a roundabout way. We hear on tech news that data has been leaked, customer accounts and info got out. There have been many high profile, well known instances. What often isn’t reported is the way in which the breaches happen. More times than not it is API’s and improper security, correct?

Topic 3 - What are the most common problems you see in organizations? What problems do folks bring you in to solve? Why isn’t a WAF (web application firewall) enough?

Topic 4 - Security, no matter the type, can be a tough sell sometimes. It’s hard to do an ROI on something that hasn’t happened for instance. What are your thoughts on this?

Topic 5 - As a followup, who is the audience that has the budget? CISO’s don’t typically come from a developer background, true?

Topic 6 - What are the typical steps on a journey towards securing APIs. Where do most folks start (assuming nothing, maybe a WAF at best) and how far does it go. Identification, automated remediation, etc.

Topic 7 - It seems every industry is being impacted in some way by AI/ML. How do you see this playing a role in the future of API security?

FEEDBACK?

Posted by
Labels: , , , , ,
Subscribe to: Posts (Atom)