Generalist AI doesn't scale
There has been a lot of talk about AI recently, and one particular point has received sigificant attention in the tech industry: The cost of training models. According to some insiders — and the market capitalization of NVIDIA — the computing power needed for AI training threatens to upend the entire semiconductor industry. This should not be a surprise: Generalist AI doesn't scale.Reduced to its essentials, the task of training a size-N model is one of hill-climbing in N-dimensional space. You take O(N) inputs, run them through your model, and after each of them you nudge the model slightly uphill towards the desired responses. You need O(N) inputs because with any less than that the model will overfit — essentially memorizing the specific set of inputs rather than generalizing from them — and for each of these inputs you need to perform O(N) computation since you have N parameters in the model to tune. End result: O(N^2) computation.
Now, there are plenty of other problems in AI — one of the largest is generating enough training data (easy enough for Chess or Go where you can have the AI play games against itself, but for general knowledge you eventually run out of textbooks) — and you can push against scaling laws for a while simply by throwing more money at them; but in the end you can't defeat scaling. You'll end up boiling the oceans.
So what's the solution? Don't do Generalist AI. Instead, we need to switch to using a pool of expert AIs. Instead of a single size-N model, split the model into k parts, each trained on N/k inputs. On sub-model learns all about medicine; another learns all about modern art. You still have N inputs, but since each of them is only used to optimize a set of N/k parameters, your training cost is now O(N^2 / k).
And yes, you lose something by doing this — you probably won't get hallucinations of modern artwork depicting polypeptides. But, as with humans, most queries can be answered by the appropriate specialist; and it's better to have a collection of experts than a generalist which is too expensive to train effectively. (One could even have a "dispatcher" sub-model which knows enough to identify which of the specialists to refer a query to.) And by reducing the training cost, you become able to build a collection of models which is larger — and smarter — than a generalist model could ever be.
Specialization isn't just for insects. It's for AIs too.
Please test: FreeBSD 13.3-RC1
I just announced the availability of FreeBSD 13.3-RC1. This is the first release candidate of FreeBSD 13.3, and if no further issues are reported will be the only release candidate; I would like to start 13.3-RELEASE builds on Friday, with (allowing time for mirrors to update) the release announcement going out on the following Tuesday (March 5th).This means there's a few days for people to do some last-minute testing and report any problems they find. If you have time to help out with testing, there are two things in particular which I'd like to see get attention:
- Wifi, especially the iwlwifi driver. Bjoern Zeeb merged a significant number of changes to the wifi and linuxkpi (which is used by iwlwifi) code between BETA3 and RC1. While these changes were tested extensively, it's still a big chunk of code — more than I would normally have wanted merged so late, but it fixed serious stability issues with iwlwifi so I thought it was worth including anyway. But I'll feel much better about the release if I know people have been testing this code.
- The installer. Most people who test FreeBSD BETAs do it by upgrading existing systems — fair enough, you test what you have. But this means that the installer doesn't get nearly as much testing as running FreeBSD systems get. So if you have a spare system laying around, please download an installer image and make sure that you can install FreeBSD 13.3-RC1! In particular, keep an eye out for any "missing" hardware or error messages about drivers being unable to reserve resources; we had a late fix to the way that ACPI devices reserve resources.
This is the first FreeBSD release I've managed since assuming the role of FreeBSD Release Engineering Lead, and the first time I've been "flying solo" (I managed FreeBSD 13.2, but Glen was looking over my shoulder for most of the release process); so it's entirely possible that I've gotten something horribly wrong. If you see anything which looks strange, please don't hesitate to get in touch — either directly (cperciva@FreeBSD.org) or by emailing the release engineering team (re@FreeBSD.org).
It's a privilege to manage the FreeBSD release process, but it also takes a significant amount of time. If you'd like to help me find time to work on FreeBSD, please consider contributing to my Patreon.
An APPR claim with Air Canada
Like most countries now, Canada has regulations requiring passengers to be compensated for flight delays under certain circumstances; Canada's regulations are called the Air Passenger Protection Regulations (commonly known as "APPR"). In May 2023, I flew to and from BSDCan on Air Canada; on the way home, one of my flights was cancelled and Air Canada rebooked me for the following day.I filed a request for compensation with Air Canada, but received a response indicating that the cancellation was due to "unforeseen maintenance" which was "required for safety purposes" and thus exempt from the compensation requirements. There were two appeal mechanisms available at this point: First, via the Canadian Transportation Agency — which theoretically reaches decisions within 90 days but has recently been reported to have a case backlog of over a year — and second, via small claims court. I opted to take Air Canada to small claims court — which in BC, means the online Civil Resolution Tribunal.
On July 13th 2023 I filed my initial claim for $1315 — $1000 of compensation for the flight delay, plus $315 because I was reboooked into "domestic business class" (regular seats) instead of the "signature class" (lie-flat seats) which I had originally booked:
On 2023-02-28 I booked flights with Air Canada from Vancouver to Ottawa on 2023-05-16 and returning from Ottawa to Vancouver via Toronto on 2023-05-21. The ticket was #014CENSORED and booking reference was CENSORED.My return flights were originally AC455 (YOW-YYZ) and AC119 (YYZ-YVR) arriving in Vancouver at 2023-05-21 17:56; on 2023-05-12 I was notified that AC119 had been rescheduled from 16:00 to 17:00 (arriving at 18:56 instead of 17:56), and on 2023-05-16 I was further notified that AC119 was being cancelled and I was being rebooked to AC117, also departing at 17:00 and arriving at 18:56. I have no claim concerning these routine schedule adjustments.
On 2023-05-21 I took flight AC455 from Ottawa to Toronto as scheduled, but shortly before flight AC117 was scheduled to board, received a notification that "flight AC117/21 May from Toronto to Vancouver has been cancelled due to an unforeseen aircraft maintenance issue. We are actively looking to find an alternate flight and will update you shortly." Two hours later, I was rebooked to flight AC101 on the following morning, and arrived in Vancouver at approximately 08:30 AM, about 13.5 hours after my original scheduled return.
I filed a claim under the Air Passenger Protection Regulations via the Air Canada website on 2023-05-26, and the website indicated that "We're sorry but this flight is cancelled due to unexpected business or operational constraints on our planned flight schedule". On 2023-05-27, I received an email from Air Canada stating that "This flight was delayed due to unforeseen maintenance that does not include scheduled maintenance or mechanical problems identified during scheduled maintenance." and denying my claim for compensation on the grounds that the flight cancellation was "required for safety purposes".
I dispute Air Canada's claim, on the basis that the aircraft scheduled to operate flight AC117 on May 21st -- registration C-GFUR -- proceeded to depart later on May 21st for Bogota, operating flight AC94. While Air Canada may have had an unforeseen safety-related aircraft maintenance issue on a different Airbus 330-300, the aircraft scheduled for the cancelled flight AC117 was in fact operating (presumably safely!) and could have been used to operate the flight as initially scheduled; it was in fact Air Canada's operational choice to cancel flight AC117 in order to use that aircraft to operate AC94 instead.
Since there was no safety-related issue preventing AC117 from operating, per paragraph 19(1)(a)(iii) of the Air Passenger Protection Regulations, I am entitled to $1000 compensation.
Furthermore, my flight from YYZ to YVR was originally booked in Air Canada "signature class", with lie-flat seats ("pods"), while the flight on which I was rebooked (AC101) had regular "domestic business class". Per paragraph 17(6) of the Air Passenger Protection Regulations, I am entitled to a refund of "the difference in the cost of the applicable portion of the ticket", namely $300 + GST.
As such, under the APPR I am entitled to compensation of $1315 plus interest and any relevant fees under the BC Civil Resolution Tribunal Act.
On August 16th, Air Canada's lawyer responded, essentially saying "we deny everything".
Next up in the CRT process is "negotiation", which basically didn't happen; I sent a message via the CRT website explaining again why I was entitled to compensation, but Air Canada didn't respond. So we moved on to the following stage ("facilitation") where a Tribunal representative scheduled a conference call between us.
That call was scheduled for 9 AM on January 10th, and as soon as it started Air Canada's lawyer spoke up: "We've looked into this further, and Air Canada is prepared to pay Colin the $1315 he is claiming". They didn't agree to pay the $75 CRT filing fee, which theoretically I could have recovered if I continued through the process; but getting $1315 was good enough for me, so I agreed to take the money and drop the CRT claim.
My lessons learned, for the benefit of any of my readers facing similar flight delays and cancellations:
- The Air Passenger Protection Regulations work!
- If you're in BC, the Civil Resolution Tribunal is fantastic. Hopefully other provinces will set up similar bodies soon.
- If your flight is cancelled, use a site like flightradar24 to see which plane was scheduled for your flight and what it is doing instead. If it spends the next couple days on the ground, it probably needed maintenance — but if it flies somewhere else, you probably have a case that it wasn't really in need of urgent safety-related maintenance.
- Be patient and don't give up.
Safe travels, and I hope I'll see some of my readers at this year's BSDCan — hopefully without any flight delays!
A Canadian payroll dependency chart
The Canada Revenue Agency publishes, on a regular basis (they're now up to the 119th edition), a document entitled "Payroll Deductions Formulas". This document contains all of the formulas needed to calculate payroll deductions collected by the CRA: Canada Pension Plan, Employment Insurance, and Income Tax. As someone running a small business in Canada and not wanting to use an external payroll provider, I implement these formulas myself in a spreadsheet.Unfortunately, like most aspects of the tax system, this steadily increases in complexity, and this year with the Canada Pension Plan including not just "base" premiums and "additional" premiums but also "second additional premiums", I needed to rewrite my spreadsheet from scratch; and also very unfortunately, the authors of the document have failed to grasp an essential detail of software: You have to calculate values before you use them. This results in a document requiring constant jumping back and forth to find where each value is calculated — before or after it is used.
To help myself keep everything straight, I decided to draw a dependency chart using GraphViz. There are 79 nodes in the graph, starting with values like "Year's Annual Maximum Pensionable Earnings" ($73200 in the 2024 tax year), and ending up with "Total payroll deductions". Note that I did not write the formulas onto the chart — I simply recorded which other values they depend upon, so someone (aka myself) looking at the chart can figure out what to calculate first without going around in circles.
Oh, and to keep things simple, I assumed I wasn't dealing with commissioned employees, employees entering or exiting the Canada Pension Plan, or anyone resident in Quebec, Nova Scotia, Yukon, or Ontario (each of which has their own special tax complications).
Here's what it looks like; click for a full size version (warning, 5627x2033
image):
Some late-breaking FreeBSD 14 breakage
I assumed the role of FreeBSD Release Engineering Lead a few days ago, and one of my first duties in the role was to write and send out the FreeBSD 14.0-RELEASE announcement. (To be clear: Glen Barber did all of the work of getting the release ready; the final bits had already been copied out to mirrors at the point that I took over.) FreeBSD 14 is a great release, but there are a few last-minute issues which deserve to be documented — probably somewhere on the FreeBSD website, but I can post to my blog much faster and hopefully we'll get these onto the FreeBSD website later.
You need to freebsd-update fetch install before you upgrade
Moving from FreeBSD 13 to FreeBSD 14 we have the unusual case of a file in FreeBSD 13 having the same name as a directory in FreeBSD 14. This is not something I ever imagined happening when I wrote freebsd-update, and my original code didn't handle it (I assumed that I could create everything new before deleting old bits). This was fixed via an Errata Notice but if you haven't installed the fix yet then using freebsd-update to upgrade to 14.0 will break.FreeBSD Update reports 14.0-RELEASE approaching its EoL
The FreeBSD Update metadata includes the release End-of-Life date; but the wrong value got inserted when the FreeBSD Update bits were put together for FreeBSD 14.0-RELEASE. Just ignore the warning; it will go away with the value being corrected along with the first Security Advisory or Errata Notice.Be careful when merging master.passwd
The default shell for root changed from csh to sh in FreeBSD 14. When you upgrade to FreeBSD 14, freebsd-update will prompt you to merge changes to /etc/master.passwd. Don't just take the new password line for root since it doesn't have a password. Keep your existing line and change the shell (or not, if you prefer to stick with csh).No PINE64 SD Card image
One of the SD Card images we normally build is for PINE64. The build failed — we're not exactly sure why, but offset.inc somehow ended up being full of NUL bytes — but we decided to go ahead without that image. The PINE64-LTS image did build.EC2 AMIs can't handle binary user-data
In order to support IMDSv2, the EC2 boot scripts were changed from using fetch(1) to get data from the EC2 Instance Metadata Service to using the newly written aws-ec2-imdsv2-get utility. Unfortunately a bug in that utility resulted in it assuming that data from the IMDS is always UTF-8 strings — which is usually true, but breaks if you provide binary user-data. In particular, if you generate a tarball and pass it to configinit it will break.If you find my work on FreeBSD useful, please consider sponsoring my work. At a certain point, time and money are fungible, and the combination of maintaining the FreeBSD/EC2 platform and my newly acquired release engineering duties adds up to a significant draw on my time.