Dran

Inference is dirt cheap in comparison. Hundreds to thousands of concurrent users can be served by hardware costing in the high-thousands to low-ten-thousands.

Training those same foundational models is weeks to months of time on tens to hundreds of millions worth of hardware.

The simpler the arbitrary string/blob parsing logic the less this happens

https://app.opencve.io/cve/?product=grub2&vendor=gnu

I agree with you that it’d be nice if the cuts were a little shallower and allowed for an encrypted boot partition, but you could still have the system reasonably secure by encrypting the data partitions and signing the entire boot process to detect and abort decryption if the boot partition doesn’t match signatures. You already have to do this with the efi partition if you’re particularly paranoid about that attack vector, so this really isn’t a new one.

If caddy is acting as a proxy for anything, you should not need to forward that port externally. Local host firewalls allowing traffic on your local network is sufficient.

Depending on your physical host layout you may be looking at an issue with nat reflection.

You have not given us enough about your topology to assist in troubleshooting.

The tl/dw is that a retrospective of history often allows us to draw parallels to the present and predict the future.

There are server chips like the E7-8891 v3 which lived in a weird middle ground of supporting both ddr3 and ddr4. On paper, it’s about on par with a ryzen 5 5500 and they’re about $20 on US eBay. I’ve been toying with the idea of buying an aftermarket/used server board to see if it holds up the way it appears to on paper. $20 for a CPU (could even slot 2), $80 for a board, $40 for 32gb of ddr3 in quad chanel. ~$160 for a set of core components doesn’t seem that bad in modern times, especially if you can use quad/oct channel to offset the bandwidth difference between ddr3 and ddr4.

I think finding a cooler and a case would be the hardest part

+1 for Niagra. It takes a few days to get used to but it’s the launcher every power user didn’t know they wanted. Lifetime purchase options and a very responsive/passionate dev

https://en.wikipedia.org/wiki/Markov_chain

Before the advent of AI, I wrote a slack bot called slackbutt that made Markov chains of random lengths between 2 and 4 out of the chat history of the channel. It was surprisingly coherent. Making an “llm” like that would be trivial.

It definitely can be disabled post-install but is much simpler to install without it at install-time, and has the added benefit of not pulling 2-5gb of other things that won’t be relevant to your use case. It’s not that the disk waste is that big of a deal, but any issues you run into will be that much easier to troubleshoot with fewer moving parts.

That wasn’t quite the takeaway I was going for. You can get a lot done on 8gb of ram. I was just trying to point out that it would probably be your first bottleneck as you started to scale out, and that you should consider using the server headless to make the ram you have go that much further.

All of those would be perfectly cromulent nodes for small containers. The first issue you’ll run into is the low ram. Some homelab projects would cause you to exceed 8gb, but the good news is if you’re using an external backend via NFS, you can always scale out (more nodes) or up(more compute per node,) later with minimal headache.

If you’re going to be memory constrained, don’t waste 1-2gb on a gui, install Ubuntu/Debian/whatever headless

Compsci labs or everywhere?

Not OP but I think this guy is remembering a scene from silicon valley, not from reality. That said it’s probably not that far off. Amazon smart devices absolutely have this “feature” in production today-- and it’s opt-out, not opt-in.

https://en.wikipedia.org/wiki/Amazon_Sidewalk

Not the same chips, but ddr5, gddr7, and hbm2 are made off the same wafers in the same plants. The issue is allocation in wafer and production time skewing towards the higher-margin items. DDR5 additionally is being made more into the server ecc variant, which companies are buying in droves for cost-efficient MOE inference.

What is the flag for this?

CGNAT does have a designated range by spec. 100.64.0.0/10, which covers addresses from 100.64.0.0 to 100.127.255.255. Technically they could be using any other private address space but it would be very uncommon in a modern ISP.

No offence: but the problem is an app forces me to trust you; a website does not. I have toghter and easier control over a web request than I do over an app, and even if an app doesn’t have these permissions today, an update or an update after a sale could trivially and silently introduce them.

A website is obvious if the deal changes-- you put up a login wall to harvest data; I stop using the site. You put trackers and ads into the UI; I block it at the DNS level.

First instinct: being an app gives me over-permissive data collection scam vibes. I will not be installing it even though I might otherwise find a website of similar capability useful.

Unfortunately it not only has to be companies, but unless you are a producer of products that are HDMI certified already your membership will be denied. It would take a lot of fuckery to make that many corporations and not have all of their membership applications be denied. Also I’m not sure that it’s even a voting democracy in the traditional sense even if you could.

I suspect the difference in experiences is more due to x11/pulse(my custom systems) vs Wayland/pipewire(bazzite) than it is any particular GPU vendor or driver branch. Which I guess is a roundabout way of saying

Maybe? Probably?

Judging by the protondb entry on CS2 I strongly suspect I would have at least the audio issue regardless of gpu.

Appreciate the recommended fixes. I did find similar and was able to work through some of the issues with CS2 but I did that on instinct, and it wasn’t until I was halfway through troubleshooting game 2 of 2 attempted that I realized it wasn’t where I needed it to be for a remote support hand-me-down.

I did briefly entertain the idea of setting up rustdesk on it but the atomic nature + Wayland made unattended (read: “help I broke it and I can’t log in”) not really viable. By the time I got to “hrm, I could probably set up a reverse ssh tunnel into my homelab for persistent support?” I decided windows was probably the play here.