I’ll give that a shot.
I’m running it in docker because it’s running on a headless server with a boatload of other services. Ideally whatever I use will be accessible over the network.
I think at the time I started, not everything supported Intel cards, but it looks like llama-cli has support form Intel GPUs. I’ll give it a shot. Thanks!
Thanks for the link. I was gonna ask if you were a writer, heh.
I agree. The tone of the ads this year felt almost like lampshading. Like if we acknowledge the problem, we’re wise to what the audience is feeling, but we’re not going to do a damn thing to address it. It’s just something that needs to be done to make the ad feel remotely relevant.
AI is scary, but don’t be afraid of our surveillance device because we acknowledged that AI is scary
AI will sell you ads. Anyway, you’re watching an ad for AI
Work sucks amirite? Why not let us unemploy you?
There’s a wealth gap. Spend money on our stuff.
And I’m not going to even link the He Gets Us ads.
It was an especially interesting case because there was a question of whether the photographer lied about who actually took the picture. So he could either claim the monkey took it an lose the copyright or claim he took it and have it lose all value.
Thanks for taking the time.
So I’m not using a CLI. I’ve got the intelanalytics/ipex-llm-inference-cpp-xpu image running and hosting LLMs to be used by a separate open-webui container. I originally set it up with Deepseek-R1:latest per the tutorial to get the results above. This was straight out of the box with no tweaks.
The interface offers some controls settings (below screenshot). Is that what you’re talking about?
Well, not off to a great start.
To be clear, I think getting an LLM to run locally at all is super cool, but saying “go self hosted” sort of gloms over the fact that getting a local LLM to do anything close to what ChatGPT can do is a very expensive hobby.
Any suggestions on how to get these to gguf format? I found a GitHub project that claims to convert, but wondering if there’s a more direct way.
GO self-hosted,
So yours and another comment I saw today got me to dust off an old docker container I was playing with a few months ago to run deepseek-r1:8b on my server’s Intel A750 GPU with 8gb of VRAM. Not exactly top-of-the-line, but not bad.
I knew it would be slow and not as good as ChatGPT or whatever which I guess I can live with. I did ask it to write some example Rust code today which I hadn’t even thought to try and it worked.
But I also asked it to describe the characters in a popular TV show, and it got a ton of details wrong.
8b is the highest number of parameters I can run on my card. How do you propose someone in my situation run an LLM locally? Can you suggest some better models?
Yo momma’s so fat, she sat on a binary tree and squashed it into a linked list in O(1) time.
Just missed Bandcamp Friday. Also, get u some flac.
zip bomb
Interesting. Didn’t know about the google books case. I agree that it applies here.
I think it’s critically important to be very specific about what LLMs are “able to do” vs what they tend to do in practice.
The argument is that the initial training data is sufficiently altered and “transformed” so as not to be breaking copyright. If the model is capable of reproducing the majority of the book unaltered, then we know that is not the case. Whether or not it’s easy to access is irrelevant. The fact that the people performing the study had to “jailbreak” the models to get past checks tells you that the model’s creators are very aware that the model is very capable of producing an un-transformed version of the copyrighted work.
From the end-user’s perspective, if the model is sufficiently gated from distributing copyrighted works, it doesn’t matter what it’s inherently capable of, but the argument shouldn’t be “the model isn’t breaking the law” it should be “we have a staff of people working around the clock to make sure the model doesn’t try to break the law.”
That study is six months old. The one I linked is from three weeks ago.
No it isn’t. Read.
That’s quite a claim, I’d like to see that.
Yet most AI models can recite entire Harry Potter books if prompted the right way, so that’s all bullshit.
I don’t get it…
Looks like it’s getting close to the Streisand kiss of death.
Also that’s not that impressive. 30 million devices is roughly one per second year round. Phone manufacturers make many times that in a year.
That Funny Feeling?