Geek

French version is available on LinuxFr Introduction This article is a follow-up to my previous entries: Self-hosting your AIs: General Principles Self-hosting your AIs: Hardware and Inference Optimization In this entry, we’re going to talk about running LLMs with little to no VRAM. In other words, we’re going to try and shove a large round peg into a small square hole. Like me, you might have noticed something strange: there are plenty of YouTube videos, LinkedIn posts, etc., telling you that running LLMs on a museum piece (like an Nvidia GTX 1060 6GB with DDR4) is super simple. They explain that it’ll run lightning-fast, work perfectly, and so on. Apparently, you just need to find the right “magic parameters,” and poof! You’ll never need Claude Fable 5 again! 🎉 ...

French version available on LinuxFr.org Introduction Comrades, digital proletarians, penguins—unite! Against the rent-seeking of proprietary AI, let us arm our GPUs! Let’s seize the means of intelligence production! Alright, soapbox aside. Let’s talk about LLM and hardware. This article is essentially a deep dive into my previous article on self-hosting AI. To get the most out of this, you should be familiar with: Linux in general Docker The PCIe bus Some basic concepts regarding self-hosted LLMs (otherwise, this is going to sting like hell). In my previous article, I suggested a few hardware solutions. Since then, I’ve gathered more data, played around much more with my toys, and—as with almost everything in the current AI world—things have moved surprisingly fast. I’ll also dive deeper into the two hardware setups I’ve tested. Each has its own pitfalls that are well worth mentioning. ...

French version available on LinuxFr.org Introduction If you believe LinkedIn gurus, AI is the new technological revolution, the new gold rush, and so on. Soon, software engineers will no longer be needed! And if you believe my YouTube recommendations, AI is the latest trendy scam, an unprecedented economic bubble inflating, and an ecological disaster ¯\(ツ)/¯. In any case, in IT professions, the subject has become difficult to avoid. For my part, after examining the problem, I reached the following personal conclusion: It’s a new tool, nothing more, nothing less. The challenge is to find how to best use it. And as an old communist bearded penguin, that means finding the most open-source tools possible and figuring out how to self-host the beast. ...

Context Summer is coming. Summer is hot. Gaming on desktop computer makes my office hotter. My laptop on the other hand doesn’t make it that much hotter. My basement is cold. Therefore my Proxmox server is cold, and it doesn’t make my office hotter. Idea Put my video card in my Proxmox server and play remotely using Steam Remote Play. Putting everything in a LXC container allows to keep things nice, tidy and separate from the host system as much as possible. The main goal for me is not security (you have to give the container access to a lot of things), but simply handling backups in a generic manner. It also allow me to snapshot the container easily before trying some dumb idea on it. ...

Introduction I currently have to teach someone how to use the Linux shell (bash). I have to do it remotely. The obvious first step is to create a sandbox virtual machine running Linux and create accounts for them on it. I have a backup of the VM so they can freely make a mess if they want to. But to guide them, I need to have a way to see what they are inputting and what they did input. I could use videoconferencing with screen sharing. But I like letting them do most of the work, so they spend a lot of time looking for info and thinking hard by themselves. So videoconferencing with them all the time would be a waste of my time. Also videoconferencing doesn’t allow me to see what they previously inputted. ...

Introduction I’m a programmer by trade and I’ve been helping my manager recruiting for quite some times. By helping recruiting, I mean separating good applicants from bad ones by having a technical interview with them. Resumes are often full of approximations and flat-out lies. The idea here is to figure out what their actual programming skills are. And I needed a tool to do that efficiently and remotely. ...

On Debian 10 (buster) and Ubuntu 20.04, switching to zsh and liquidprompt, the quick and easy way: 1 2 3 4 5 sudo apt install zsh liquidprompt chsh -s /usr/bin/zsh # restart your session liquidprompt_activate # close your shell and re-open it

Here is the recipe I’ve made recently to integrate Kivy 2.0.0 in Yocto (dunfell). It’s probably far from perfect, but I guess it can help other Yocto+Kivy users like me. Note that I had to patch the setup.py of Kivy to work around a bug. This bug has been fixed now, so this patch won’t be useful for future versions. SUMMARY = "Recipe to embedded the Python PiP Package Kivy" HOMEPAGE = "https://kivy.org/" LICENSE = "MIT" LIC_FILES_CHKSUM = "file://LICENSE;md5=f0c851d60495c7e27225a819e179208a" PYPI_PACKAGE = "Kivy" USE_WAYLAND = "1" export USE_WAYLAND inherit dos2unix pypi setuptools3 SRC_URI += " \ file://0001-setup.py-Fix-get_modulename_from_file.patch \ " SRC_URI[md5sum] = "410d3968c7f45171285bcac76bbad265" SRC_URI[sha256sum] = "d25e44eb44e43762b2fd0c5874e51954e0f1181fd9800d8a6756be6d084812d8" DEPENDS += " \ gstreamer1.0 \ gstreamer1.0-python \ libsdl2 \ libsdl2-image \ libsdl2-mixer \ libsdl2-ttf \ pango \ python3-cython-native \ " RDEPENDS_${PN} = " \ gstreamer1.0 \ gstreamer1.0-python \ libsdl2 \ libsdl2-image \ libsdl2-mixer \ libsdl2-ttf \ pango \ python3-docutils \ python3-fcntl \ python3-image \ python3-kivy-garden \ python3-pillow \ python3-pygments \ " do_install_append() { # drop the examples # XXX(Jflesch): by default they shouldn't have been built by # Kivy's setup.py. Therefore they shouldn't have been installed # either. Yet somehow here they are. rm -rf ${D}/usr/share } Kivy-garden is also required by Kivy, so I also had to make a recipe for it. You can download the complete recipes and patches for Kivy and Kivy-garden here: yocto_recipe_kivy.tar.gz. ...

Use Ubuntu 20.04.1 (Raspbian is only available as armhf images ; we need arm64). In /etc/modules, add libceph and rbd Install K3s Clear the disks you want to use as Ceph storages As root: 1 2 3 sudo apt install ceph-common git clone https://github.com/rook/rook.git cd rook Switch to the raspbernetes images instead of the default ones. Currently the default ones are not all built as multi-arch yet and therefore don’t all work on arm64. ...

My girlfriend and me have bought the very same electric kick scooter, the Segway Kickscooter Ninebot ES-2. We have done the very same trip at the same time. Both scooters are brand new. This showcase very nicely the importance of weight regarding electric kick-scooter autonomy. Beware: This electric kick-scooter requires you to install an application on your phone. On Android, the application is available on Aptoide and works without Google services. However this application is 100% proprietary, poor quality, and clearly from China. It will publish the distance you travel somewhere without asking you first. ...