EFF is competent, and there’s plenty of legitimate open source projects with reasonable name recognition. Also, like, android (google) and vscode (ms). Plenty of headline products that are open source and want to keep running. I think there’s interests on both sides of this one.

Could you walk me through the damage that this mistake does to well-and-proper open source developers?

I don’t think it is likely that they would be harmed by things like new regulation on ‘public source code’ projects? Anybody looking to hire a developer who confuses these two wasn’t serious/relevant in the space. Open source developers do not, at least in my media diet, seem to get a lot of airtime or respect for their work (or money, for that matter). And the places they do get these things are sufficiently literate that I’m not concerned.

Could you help me out with this claim? What’s the model of harm?

GenAI is an averager (as all empirical models are), so it tends towards predicting the mean of whatever it’s trained on (in a given context).

Perhaps this sentence is not what you wanted then? The mean of whatever it’s trained on (that is, the training data) need not be the mean of the trained LLM function. This is because we use complicated loss functions to update our LLMs, stochastic processes in training, and that we change those functions after training on input data by using RLHF and other models (eg, via using a mixture of itself to get better outputs with the boosting algorithm).

You are right that once we fix an LLM’s parameters, at temp 0 they are deterministic, though they are recursive functions so it’s weird to talk about a long string of outputs at temp 0 as the ‘mean’ of the LLM; high temp outputs won’t ‘regress’ to this temp 0 output over time, for example. There also may be hallucinations or mistakes that are common at temp 0, but extremely rare at all other temperatures (because these mistakes only get made when it looks at its own temp 0 context). So I’m not sure the deterministic output string(s) are particularly useful.

(An analogy that I suspect isn’t very good: the butterfly effect, adding a single small air perturbation can lead to drastically different deterministic weather simulation results. This is very much the norm for LLM outputs)

(They are somewhat more than a neural network; the ‘attention’ layers add something pretty strange. And output temperature isn’t implemented as a noise term, but this is a fine way to think of it.)

(There is a separate claim about genAI producing the average/typical behavior from its inputs; this isn’t true for RLHF and boosting reasons, have a gander at boosting and wisdom of the crowds for how you can take a cheap source of mediocrity and create something much better. It is true that genAI outputs feel unoriginal and omnipresent, as so very many folks are throwing around genAI outputs everywhere. But I think that’s not a property of the statistics, rather of the economics/behavioral science.)

So I think you’ve landed on the problem that I have with student use, and that the article has with scientist default use. It is very bad for folks to use AI to try to do the knowledge work directly, yet people are making this mistake constantly. YSK: it’s much better when you make AI increase the friction of knowledge work. (note neither are claiming that this use is particularly good; we’re doing damage control.)

I definitely notice sloppy thinking when I see it in someone else’s review of my papers. You certainly pick it up when you read climate-change-denialists. Experts are great at noticing sloppy thinking when it disagrees with them. That is the relationship you want with genAI if you are using it for knowledge work. Make it disagree with you, so you notice where it is very sloppy (and sometimes where you’ve been sloppy because the bad thinker is kinda right).

As I’ve been messing around with it, I am becoming a fan of ‘hallucination -> new context’.

(Why is consciousness required to tell true/false statements? I think I’d even say ‘that sign is dishonest/wrong/a liar’, regardless of the fact that the sign is very much dead.)

Knowledge work is generally pretty expensive; it’s done in dense places, uses a lot of air travel, has pretty big long-term consequences. Even the high-end estimates on AI costs are probably worth modest gains for knowledge workers.

Idk about you, but my mentors and peers are all busy stressing over the polycrisis.

Could you flesh this out for me, I’m not sure I understand? I think you’re saying:

1. experts are not well posed to catch biases and untruths generated by genAI in their (research work)
2. because as an academic climate scientist, your day-to-day work is spent on new climate science (and not established climate science).

I don’t see why (2) implies (1), but I agree with (2). As I intended it, (1) is my main claim. I follow it up with

3. if an expert cannot spot a bias or flaw in genAI output, then they wouldn’t catch it from a peer either.

I don’t see how (2) helps with (3) either.

lol (This is why we can’t have nice things)

If you’re doing knowledge work, you are an expert. You’re very likely to catch biases and untruths (and those you don’t catch, humanity couldn’t catch. These concerns are just as applicable to a human by themselves.)

Thoughts on the study from the article? link

Sometimes knowledge work is about learning. That’s also a nice time to skip AI. But I can see from the data that student’s aren’t doing that, so we harm-reduce.

(Though I’ve recently been told that it uses Bing’s index, so real searches may struggle.)

humerously, web archive also gave me a pop-up about my blocker. But ads aren’t served on web-archive? so I just turned the blocker off.

Yes it’s the church; lots of missionaries go international. Most lds teens get the document ahead of their call. And more will vacation abroad if a parent has an international mission.

If you are elsewhere, but want to do something similar:

https://en.wikipedia.org/wiki/List_of_assets_owned_by_Nexstar_Media_Group

https://en.wikipedia.org/wiki/List_of_stations_owned_or_operated_by_Sinclair_Broadcast_Group

(seems in the seattle area there is also KUNS-TV?)

I’m so happy for you! Today you learn the world has gotten better! https://ourworldindata.org/life-expectancy

Just improved hygiene and treatments for babies have essentially doubled humanities life expectancy in the last century. But even restricting to older cohorts (ie, assuming you live to 70, how many more years?), life spans have grown. The vast majority of people are not wealthy, so these gains are very much seen by the poorer folks.

Just noting that image gen is done by s different architecture than an LLM, and the environmental impacts are not the same (in particular, much much less energy use than having a human make one image, and substantially less water use than going to the toilet).