Apples AI struggles

On Feb. 14, Bloomberg News reported that Apple was struggling with bugs and engineering problems on its planned artificial intelligence tools for Siri.

"Apple Inc.’s top executive overseeing its Siri virtual assistant told staff that delays to key features have been ugly and embarrassing, and a decision to publicly promote the technology before it was ready made matters worse."

"...the company’s woes boiled over last week, when it acknowledged publicly that critical features would be delayed indefinitely."

"Despite not being ready, the capabilities were included in a series of marketing campaigns and TV commercials starting last year."

"Walker said the decision to delay the features was made because of quality issues and that the company has found the technology only works properly up to two-thirds to 80 per cent of the time — which is to say it doesn’t work every one out of three times. He said the group “can make more progress to get those percentages up, so that users get something they can really count on.”

Walker ended the meeting upbeat, saying that Apple will “ship the world’s greatest virtual assistant.”

He showed examples during the meeting of the technology working: It was able to locate his driver’s license number on command and find specific photos of a child. He also demonstrated how the technology could precisely manipulate apps via voice control. It embedded content in an email, added recipients and made other changes." 

This is very unusual for Apple. A company known for shipping great consumer products which just work. Its also unusual for them to announce in marketing campaigns products which are not yet ready.

But what is the root cause of the problem? "the company has found the technology only works properly up to two-thirds to 80 per cent of the time"

What technology? Its got to be the model. We all know Apple can build end to end systems which are highly reliable. A failure rate of one third is unheard of in Apple products. So it must be the model which is unreliable approx 30% of the time. Wow! 

Besides Waymo automonous driving, where else do we see AI integrated into consumer end products? 

ChatGpt, Glean, Customer Service chatbots and AI developer tooling are examples. noting: Google has released AI Mode (different from AI Overview) for search which is simiar to chatGpt. And some other interesting announcements this past week in Google IO. 

But beyond that, not much, yet. So maybe Apples struggles to build AI enabled consumer devices are not just about Apples execution. 

Search is allowed a higher failure rate, than say identity verification, which can never fail. Likewise code prediction. In both cases when AI is wrong, knowledgeable humans can adjust. If the model returns wrong or partially wrong search results it's usually not a big deal. But a 30-20% failure rate is a real hurdle for the next gen of AI products.


Comments

Post a Comment

Popular posts from this blog

angularjs ui-router query string parameter support

angularjs ui-router is pretty cool, we use it in our app now but have not yet used query string parameters (e.g. ?p1=age&p2=height). But now we need to. we're now adding a feature which is basically a report generator and its a good use case for query string parameters once on the page then you're basically in that ui-router "state" users can select filters etc. from dropdowns which will reload the data but should not destroy/recreate any controllers or ui widgets (we stay on the page and do not state transition) yet we do want to update the url for deep links and back button support we don't know the complete list of possible filter choices, but know it should be data driven and will change by report  urls to support this report feature are will be something like thus /reports  /reports/sales  /reports/sales?region=west&time=weeks so we need to use query string parameters Below are some of the things I discovered with ui-router query strin...
Read more

react-select stacking order bug, z-index, layers and stacking

Image
Had to fix a tricky bug related to react-select recently (our Dropdown component wraps react-select). Here's the how the bug manifested. Dropdown does not overlay dynamically added content below it Variations of this have been reported since 2017. But I tried all the suggestions in the comments and none worked for me. I also tried wrapping the Dropdown in a div and changing z-index to be higher as well as making absolute position but no success. Changing the elements order in the html source and then reordering using flexbox order does work but it breaks the tabbing flow and is not recommended from an accessibility standpoint. One issue is the Dropdown is in a container and is constrained by the containers z-index. No matter how I change the Dropdowns z-index, its container is stacked below the next in stacking order. Google Chrome debugger layers tool provides a cool way to visualize layers. Could be better but helps, check it out below. You have to look closely but you can see h...
Read more

typescript notes (typeof, keyof, ReturnType, Parameters, Extract)

  typeof extract a type from runtime code, typeof "make a type of it" (i.e. from it) e.g.   typeof "Hello world"  => string only legal on variable names can be used with functions or any other expressions aside: typeof vs keyof: keyof takes a type and creates a string/number union of it's keys. So keyof creates a new type from a type ....whereas typeof creates a type from an expression. good post Given a function: const myFunc = (name: string) => {   return `hello ${name}`;    }; type the function into MyFunc type MyFunc =   typeof myFunc; type a function return type (using built in ReturnType ) type MyFuncReturn = ReturnType<typeof myFunc>;  // string When the function changes, the types change! You can also type the parameters using utility type  Parameters      type MyFuncParams = Parameters<typeof myFunc>;  // string but Parameters will return a tuple (array), so you could index result like...
Read more