The idea of installing a software update on your fridge already feels strange, let alone focusing on improving its AI capabilities. But that’s exactly what’s happening with Samsung’s line of Bespoke refrigerators this week, and to my surprise, this piece is making huge strides in providing a truly useful machine readout for today’s icebox.
As a quick recap, Samsung has offered powerful AI features such as automatic food recognition and meal planning in its Bespoke refrigerators for a few years already. However, as I discovered after reviewing its flagship model late last year, the company’s AI capabilities are still a work in progress. Previously, the fridge could see about 60 types of fresh food (such as fruits and vegetables) alongside about 50 other packaged goods such as yogurt or popcorn. That sounded like a good start, but when you consider the sheer number of different items you can find in the average grocery store, it was far from finished. In addition, you had to enter additional data such as the number of items or when something was added for the first time, which made the concept of AI-assisted grocery tracking more tedious than I would have liked. I don’t know about you, but I usually don’t want to type on my fridge and I’m sure Samsung’s engineers agree, which is probably what brought this major update to its Bespoke fridge software.
So what’s new?
The biggest change is that Samsung is adding support for Google Gemini, which has several important implications. By combining Samsung’s existing on-device object recognition with Google’s cloud-based models, the total number of foods that can be identified increases from more than 100 objects to more than 2,000. Now this means you will need to connect the fridge to Wi-Fi, but when you consider the number of other smart features it supports such as calendar integration and video playback, that is not a big question.
Another update is that Samsung is using Gemini to extend voice controls, allowing users to ask the refrigerator to change things like device settings, check details like when the water filter was last changed or help with troubleshooting. And depending on the situation, the refrigerator can even play a lesson on how to solve the problem.
Additionally, for complex or difficult-to-solve problems, Samsung is introducing what it calls Reliability AI, which is designed to monitor fridge components or help identify faults before they get worse. Or in cases where the device needs to be operated, AI can provide detailed information to agents while allowing them to fix certain things remotely.
For example, a Samsung representative told me that if a customer calls and says the cubes from the ice maker are coming out lumpy and stuck together, Reliability AI can allow agents to reduce the amount of water added to the ice tray — all without needing to come to your home. Sadly, Samsung says that while the fridge will monitor and track device health metrics, owners will need to give express permission for maintenance workers to access that data. Meanwhile, if a problem needs to be worked on manually, Samsung says that by sharing this data with repair technicians, it allows people to identify and solve problems faster rather than arriving without context and diagnosing problems from scratch.
How does this work in the real world
My test unit didn’t experience any mechanical problems in the eight months I used it, so I wasn’t able to test the Samsung Repairability AI. That said, after testing it first at Samsung headquarters, I had a chance to use the first version of the Bespoke line’s new software two weeks ago — including its improved support for cloud-based object recognition — and the improvements are profound. Even after using it for a while, I’m still amazed at how many types of food it recognizes. Deep in the back of my refrigerator, I have a can of Bull Head Shallot Sauce, an ingredient from Taiwan that is used almost exclusively in Asian dishes. However, the AI had no problem recognizing it, automatically tagging it and including it when it was first added to the fridge’s AI Food Manager.
In addition, the system is now much better at identifying brands and calculating the amount of specific ingredients to create a more detailed list. It can distinguish between Diet Coke and Coke Zero while accurately noting that there were multiples of each. And although the fridge usually has to install a cloud-based server elsewhere to help identify the various items, the results appear quickly, often in less than a few seconds. And with certain foods like avocados, I’m glad the fridge keeps track of how long you’ve had and will issue a notification that it might be nearing the end. Granted, it’s not always perfect, but what I really need is a reminder to check things out and it does just that.
Sam Rutherford of EngadgetI also noticed that the refrigerator now remembers when you often take out certain foods and asks if you want to add that item to your shopping list. It’s a great reminder to fill in the basics you use regularly and it’s done in a low-friction, non-irritating way. From there, you can just check your phone when you’re in the store instead of having to organize a list every week. Also, because the fridge does a better job of seeing and keeping track of what’s inside, it can provide better suggestions about recipes you can cook using ingredients you already have on hand.
That said, like most current models, the AI doesn’t always hit every detail. For example, I was impressed at first when it automatically labeled a tub of fake cheese as “Philadelphia Plant-based,” until I realized that the label was incomplete and the AI was simply reading what was written on the lid and didn’t have the intelligence to complete the description accurately. Don’t get me wrong, it provides more than enough information to help me figure out what’s in the fridge when I look at Food Manager. It is not yet clear.
Outlook and things that still need work
One problem with this update is that like most AI services today, Samsung’s new software can be overconfident or prone to missing things. One time, when my wife was putting something back in the fridge, the algorithm took a picture of a brightly colored bandage on her finger and labeled that as a veggie, which is very unlikely. Sometimes it seems like it’s just a guess. But I would argue that going from about 100 to 2,000 identifiable objects is a welcome improvement even if there are limitations.
Another strange thing is that even though Samsung uses Google’s AI models for many of the new features of the fridge, you won’t see any obvious callouts from the Gemini inside the device itself. That’s a bummer because Bixby is still the only digital assistant you can use and talk to directly.
I’ve said before that Samsung’s AI food recognition is a work in progress and I think that’s still true. With this latest update, the company is one step closer to delivering on the promise of a fridge with truly usable AI features. What once sounded like a promising tech demo has turned out to be a useful tool for tracking your groceries, even with a hitch here and there.
.jpg "TrickMo Android bank accepts TON blockchain for covert comms")
