Andrew Ng - Agentic AI talk

This is a good talk from Andrew on Agentic AI

He packs a lot of info and ideas into this talk. 

Andrew starts with his view that "AI is the new electricity" - because its a general purpose technology to allow building of new applications

Andrew provides an overview of the current AI tech stack, but adds a new layer: an "agentic orchestration layer"

  1. applications: credo, woebothealth, workera, meeno etc.
  2. new agentic orchestration layer: langchain, crewai etc.
  3. models: openAi, anthropic, llama
  4. infrastructure: aws, google, azure
  5. chips: nvidia, amd etc.

Most of the opportunity will be in building new AI applications (even though models are currently getting the most attention). Andrew said he is "most excited" by Agentic AI (AI agents). Agentic AI is an automonous workflow which uses advanced reasoning to solve problems by reasoning, trial and error and iteration. "zero shot" when you ask a question and get back an answer e.g. original chatGpt. Agentic workflow takes longer, refines the answer, iterates on "drafts". Agentic workflows can provide much better results.

Andrew highlights 4 major design patterns in Agentic workflows:

  1. Reflection
    • take answers from llm and feed it back in to get an improved answer i.e. the llm is critiquing its own output
    • Andrew talked about having 2 lls: one to write the code and the other to critique the work, both collaborating and iterating
  2. Tool use (api calls)
    • llms can make api calls
  3. Planning (decide on steps to ask)
    • llm takes a complex request and pick a a sequence of actions to execute to solve the request
    • e.g. request "generate an image when a girl is reading a book and her pose is the same as the boy in the image example.jgp then describe the image in your own voice"
      • Andrew explained this could be broken into multiple steps to deliver on the complex task
        • pose determination
        • post to image
        • image to text 
        • text to speech
    • "take a task and break into subtasks"
  4. Multi agent collaboration
    • can choose models which are specialized to a task e.g image to text may be best done by a "vit-gpt2 model" whereas text to speech may be best done by a "fastspeech model"

Andrew gave a demo of an app he developed va.landing.ai which uses agentic workflows for ai tasks. Effectively an agentic orchestration layer for video processing. He showed how an image was uploaded and then user can ask a question about the image "how many players in the image". The agentic workflow solved the problem and also provides python code for the user to use processing many images.

  • Andrew said this could be used to access a companys videos and process, extract value
  • Could process videos to extract metadata and build apps on top of that data. e.g. you could search a library of videos for matches e.g. "find all airborne skiers un the video"

Supervised learning which could take months to 1. assemble the data 2. train the AI model on the data and 3. deploy the model

Is now being replaced by much faster LLM based development which may take days/weeks to create a prototype and test it. Being able to build prototypes quickly allows teams to try lots of ideas quicly

  • The bottle neck is now moving to Integration and Dev Ops.


Comments

Post a Comment

Popular posts from this blog

My Reading Lists

I had some time over Christmas and decided to organize my reading lists. Interesting how some time away from daily work allows one to raise up your head   Uncle Bob on Principles of OOd See also book: Agile Principles, Patterns, and Practices in C# Available on Safari Books online. Algorithms in a Nutshell Nine   Algorithms   That Changed   the Future Programming Pearls book: http://www.cs.bell-labs.com/cm/cs/pearls/toc.html 97 Things Every Programmer Should Know Seven L a nguages http://pragprog.com/book/btlang/seven-languages-in-seven-weeks Codys book : http://www.domenlightenment.com/ Principles of OO programming: http://www.nczonline.net/blog/2012/12/18/now-available-principles-of-object-oriented-programming-in-javascript-beta/ Object Oriented programming with Stefan Stoy... Chrome V8 : http://code.google.com/p/v8/ https://developers.google.com/v8/ https://developers.google.com/v8/design - read this Mozilla Javascript page ...
Read more

angular js protractor e2e cheatsheet

Many from the protractor test suite category detailed task syntax example load page load a page browser.get('yoururl') browser btn actions browser back/forward browser.navigate().back() OR browser.navigate().forward() access elements on page See Protractor API list and examples from Protractors test suite access by angular ngmodel name (must match ngmodel in the html) this is the preferred approach element(by.model('person.name')); access by the css id on the page e.g, #user_name (but this can be brittle) element(by.id('user_name')) access angular binding to get generated value i.e. for {{}} element(by.binding('person.concatName')); access angular repeat list data (must match repeat stmt in the html) See Protractor test suit l...
Read more

react-select stacking order bug, z-index, layers and stacking

Image
Had to fix a tricky bug related to react-select recently (our Dropdown component wraps react-select). Here's the how the bug manifested. Dropdown does not overlay dynamically added content below it Variations of this have been reported since 2017. But I tried all the suggestions in the comments and none worked for me. I also tried wrapping the Dropdown in a div and changing z-index to be higher as well as making absolute position but no success. Changing the elements order in the html source and then reordering using flexbox order does work but it breaks the tabbing flow and is not recommended from an accessibility standpoint. One issue is the Dropdown is in a container and is constrained by the containers z-index. No matter how I change the Dropdowns z-index, its container is stacked below the next in stacking order. Google Chrome debugger layers tool provides a cool way to visualize layers. Could be better but helps, check it out below. You have to look closely but you can see h...
Read more