Recently I had to cancel an order. The support for the company was an LLM bot. I accidentally mistyped a number in the order id. It accepted it anyways refunded every order on my account that includes the product I wished to cancel.
I tired to get to a human to correct the mistake and couldn’t their phone number is an LLM bot their only chat is an LLM bot.
It use to not be. But now I’m sitting here the order in my hand cause the bot didn’t cancel it. But like 30 orders from the last few years have all been refunded to me.
I tried to reach em a few more times but couldn’t and it’s been like a month. I just have like 2 grand usd that I shouldn’t and no way to give it back.
I hope you saved what you could from that exchange, as well the attempts to contact them. If they ever notice, their AI mistake will become your problem to deal with, (and the kind of news story to end up on a Steve Lehto video).
If that happened to me, I’d have a chat with my bank, “please help me return this money to where it came from, it was payed in error. They have no way to contact a human and I don’t want them to accuse me of fraud down the line”.
I don’t know, more and more of those teams these days are being headed up by the same folks. Most on the ground, in the weeds know what not to do but the ivory tower keeps building more and more floors without ever updating the foundation.
Heh. Watched an old episode of Scorpion yesterday. The one with the armed hostage-takers who just had the one demand to the social media data mining company, to delete all the data they’ve mined. I amused myself a lot, by uttering “I like these guys”.
So, I’m currently developing a chatbot for my company. If an LLM needs to do something, a developer must build a tool. It just so happens that this tool that was built did not take traditional security into account. Really it should only be using the tools already built for users, but it seems the Jr. Devs that have been replacing seniors do not have the sensibilities yet.
The entity being manipulated is not human so I would not classify it as social engineering, even if similar techniques are used (help me my grandmother needs info).
It was largely overblown due to it getting banned. It was also published in the height of the Vietnam War, when the big evil communists were coming to brainwash your children into eating each other. It has a lot of blatantly incorrect info, which could be outright “blow up in your face” dangerous to anyone attempting the things in it. It’s not all wrong, but certain recipes have incorrect info that could easily lead to accidents.
Also fair warning, the UK will give people hard prison time simply for owning it. So maybe keep that shit onion-encrypted if you’re in the UK.
I linked to the Wikipedia article, not the handbook inself. And more for the (obsolete) phreaking content than the (highly dangerous) explosive content.
Why would the LLM tool have access to send recovery emails to non account verified emails at all?
That’s insane.
Recently I had to cancel an order. The support for the company was an LLM bot. I accidentally mistyped a number in the order id. It accepted it anyways refunded every order on my account that includes the product I wished to cancel.
I tired to get to a human to correct the mistake and couldn’t their phone number is an LLM bot their only chat is an LLM bot.
It use to not be. But now I’m sitting here the order in my hand cause the bot didn’t cancel it. But like 30 orders from the last few years have all been refunded to me.
I tried to reach em a few more times but couldn’t and it’s been like a month. I just have like 2 grand usd that I shouldn’t and no way to give it back.
So that’s fun.
Would sincerely love to know the name of the company. You know, to avoid them. Yup. I’m sure that’s the reason.
I wonder how long you need to keep that money aside before you can spend it?
That’s amazing
Maybe I ought to be taking more advantage of this era of rampant incompetence
I hope you saved what you could from that exchange, as well the attempts to contact them. If they ever notice, their AI mistake will become your problem to deal with, (and the kind of news story to end up on a Steve Lehto video).
If that happened to me, I’d have a chat with my bank, “please help me return this money to where it came from, it was payed in error. They have no way to contact a human and I don’t want them to accuse me of fraud down the line”.
Who else is going to have access to it when you keep laying off all the people?
Because AI bros are incredibly deluded about both the capability of AI, and by extension their own capabilities using AI>
Because one of the biggest companies on the planet that has issues with account takeovers clearly has no internal red team working on this stuff.
I guarantee they do have a red team that most likely flagged this as an obvious and severe risk. It was ignored by suits experiencing AI psychosis.
I don’t know, more and more of those teams these days are being headed up by the same folks. Most on the ground, in the weeds know what not to do but the ivory tower keeps building more and more floors without ever updating the foundation.
“one of”
should’ve asked it to delete the database instead, why else would it have that level of permissions.
Heh. Watched an old episode of Scorpion yesterday. The one with the armed hostage-takers who just had the one demand to the social media data mining company, to delete all the data they’ve mined. I amused myself a lot, by uttering “I like these guys”.
Oh, man, I hope someone tries this.
Little Tommy Drop Tables.
Drop Table suddenly becomes the newest baby name fad.
This isn’t even a hack, it’s just poorly written endpoints.
Would you consider phreaking equivalent to hacking? This is AI phreaking.
It’s not phreaking. Social engineering.
So, I’m currently developing a chatbot for my company. If an LLM needs to do something, a developer must build a tool. It just so happens that this tool that was built did not take traditional security into account. Really it should only be using the tools already built for users, but it seems the Jr. Devs that have been replacing seniors do not have the sensibilities yet.
The entity being manipulated is not human so I would not classify it as social engineering, even if similar techniques are used (help me my grandmother needs info).
Kinda.
If you designed a publicly addressable system since 1985 and didn’t design it for security then you’re asking for it.
TIL about phreaking
Then this is also probably new to you
The Anarchist Cookbook
I need to set aside some time to read that although I’m not an anarchist myself.
It was largely overblown due to it getting banned. It was also published in the height of the Vietnam War, when the big evil communists were coming to brainwash your children into eating each other. It has a lot of blatantly incorrect info, which could be outright “blow up in your face” dangerous to anyone attempting the things in it. It’s not all wrong, but certain recipes have incorrect info that could easily lead to accidents.
Also fair warning, the UK will give people hard prison time simply for owning it. So maybe keep that shit onion-encrypted if you’re in the UK.
Thanks for the warning on the blowing up! Well, I’m certainly not in authoritarian UK.
What is '69 ?
I linked to the Wikipedia article, not the handbook inself. And more for the (obsolete) phreaking content than the (highly dangerous) explosive content.
It’s because they move fast and break things. They think that makes them cool.
Let’s mix these chemicals and see what happens. No funds for lab coats or protective glasses. We got a bottom line to feed.
Hold on, do you expect Facebook to pay a human to deal with the inventory? Come on now.
It’s not insane. It’s advanced!
I tried this and couldn’t get it to work. Disappointed.