Dario has a history of making well calibrated claims about AI progress. Can you point to examples of him making unreasonably hyped up claims in order to fundraise?

This makes sense. Measures of IQ supposedly become unreliable at the very high end of the scale too, but it is still useful across the general population.

If you took a random sample of the population and got them to compete in a random sample of Olympic events, success in one event would likely to be correlated with success in other events, but some would be anti-correlated. For, example very obese people would be bad at a lot of them, but maybe good at powerlifting. Very fit people would be good at a lot of them, but maybe not all.

There would probably be some factor that was correlated with success across all the events, although it wouldn't be a perfect correlation. This could be called a general factor of athleticism.

Do you think this factor wouldn't show up if we actually did this in the real world and ran a statistical analysis?

Michael Jordan was much fitter and better trained than the vast majority of people ever are, but that doesn't correspond to a remarkable degree of talent in baseball in particular.

I was assuming the Athleticism Quotient simply measures how athletic people are and makes no assumption about the cause. A high AQ would likely correlate with innate athletic ability but it wouldn't measure it directly.

Maybe it is only a weak correlation due to training effects, environmental factors, etc. But I would still expect to see some general factor of athleticism to emerge.

In that sense, a "general factor of athleticism" is about as real as a "general factor of fitness" across which you can compare unrelated species in completely separate ecological niches.

Some athletic activities will be anti-corrleated, but most will be correlated.

Deciding what to include in the measure might be difficult, but it probably won't matter much because most activities will be correlated.

Some people are just much fitter and more athletic than others.

This probably won't be that hard to measure, you can just look at people and get a reasonable idea of how athletic they are (e.g. are they obese?). It's really obvious that there are large differences in athleticism between people and that they could be measured.

Whether this is due to innate ability is another question.

Being an ok-ish minor league baseball player means you are better than 99.9% of people at baseball. That seems pretty significant and pretty strong evidence for a general factor of athleticism.

If you scored peak Michael Jordan on a representative sample of athletic activities and tried to calculate an Athleticism Quotient, he would score absurdly highly.

Terence Tao is one of the world's greatest living mathematicians. If he thinks there's a good reason to expect AI to be able to solve difficult math problems in the near future, there's probably a good reason for that.

The main difference for me is the kick pedal, which is much worse on the Debut. Otherwise, it's pretty much the same, but smaller. The kick pedal does suck pretty bad though.

USB B into the drums then USB A into the computer?

Yeah, pretty much. Then map the controls in Clone Hero

He has written about the ontology of psychiatric conditions here: https://www.astralcodexten.com/p/ontology-of-psychiatric-conditions-34e

We're probably not far from being able to fine tune cutting edge LLMs like GPT4 or Claude 3 on thousands of charts and have them automate some of those tasks.

I've set a shortcut that opens my web browser when I hold down 1 on my lg remote, and set the web browser to open the stremio web app by default. I find it just as convenient as an app by doing this.

Honestly, his twitter feed is really boring. It's almost entirely about how much he enjoys programming. Really doubt that there's more to it.

Or it will just continue to do the thing that it has been trained to do. If you keep scaling GPT4 to be more and more intelligent, will it just stop trying to predict tokens at some point? Through what mechanism can it's reward function change?

These responses are, for the most part, terrible.

For the record, I think AI is probably going to kill everyone, but this is the best argument I've come across against that position:

AI Optimism - AI is easy to control

If you makes a machine that is superhumanly good at achieving some goal like predicting the next token, how will ever come to have any other goal? It's goal is to predict tokens. If it started to to try to help humanity flourish, it wouldn't be predicting tokens and would fail to achieve the goal it's been programmed to achieve. This idea is sometimes called Goal Content Integrity.

AIs just do whatever it is they are trained to do. Being really good at predicting the next token or pixel doesn't lead to more moral actions. If an ASI has to choose between perform a moral action or doing what it was trained to do (predict the next token), it will do whatever it was trained to do. It's not even a genuine choice.

This isn't an argument, it's just sheer anthropomorphism. Currently, these systems are trained to predict tokens and pixels, so they predict tokens and pixels. Future superintelligent systems will also do what they are trained to do (for good or bad).

I don't have any particular fondness for credentials and think that large portions of academia produce fake knowledge. I also agree that knowledge in one area doesn't mean you automatically have knowledge in a completely different area of knowledge, e.g., aerospace safety and understanding global supply chains.

But I think it is true that people who are knowledgeable in one area are more likely to be knowledgeable on adjacent topics, e.g., aerospace engineering and aerospace safety. Do you think this is false? You avoided answering this question.

Or do you think knowledge about risks from AI is not adjacent to knowledge about AI?

Also, if people who are knowledgeable about AI don't have any special insights into risks from AI, who does? Is it only people who have spent decades specifically researching risks of doom from AI that have any insight?

Because I've got bad news for you, the people who have spent the most time researching AI extinction risks have even more pessimistic expectations about AI doom than the average AI engineer.

This argument is way too general and the analogy to police seems weak. Do you think a typical aerospace engineer has a better or worse understanding of aerospace safety than the average person? Maybe they actually have a worse understanding for...

checks notes

...irrelevant psychological reasons (with likely negligible effect sizes in this context).

I meant signalling as in signalling theory in evolutionary biology, which is not typically associated with cancel culture rhetoric.

If you're that talented and successful, that kind of signalling is unnecessary at best and probably just cringe.

I was a bit confused when I saw this morning how much my power was going to go up.

But it's absurd that my partner and I were getting a subsidy when we have a pretty good combined income and no kids. Seems very very poorly targeted. I imagine there are many better ways to help low income families and our tax dollars would be better spent on those programs instead.

Probably more specific than that. Attributing this to the economy having misaligned incentives in general doesn't seem very useful without explaining more specifically how the incentives of individuals and groups within the economy are misaligned.

Generally, this is because it is agents who respond to incentives and the economy in general is not usually considered a coherent agent that can respond to incentives.

Could you explain why exercise is underused due to misaligned incentives?

Exercise is basically ubiquitously recommended and there are large industries with strong incentives to get people to exercise more.

That was a rather unpleasant read, but definitely relevant.

To be honest, I'm not familiar with the literature at all. I got the impression that MAOIs are not very well researched from this post: https://slatestarcodex.com/2015/04/30/prescriptions-paradoxes-and-perversities/

The big problem is: drugs go off-patent after twenty years. Drug companies want to push new, on-patent medications, and most research is funded by drug companies. So lots and lots of research is aimed at proving that newer medications invented in the past twenty years (which make drug companies money) are better than older medications (which don’t).

I’ll give one example. There is only a single study in the entire literature directly comparing the MAOIs – the very old antidepressants that did best on the patient ratings – to SSRIs, the antidepressants of the modern day4. This study found that phenelzine, a typical MAOI, was no better than Prozac, a typical SSRI. Since Prozac had fewer side effects, that made the choice in favor of Prozac easy.