ap.io/blog

On Entropy and the Second Law of Thermodynamics

What's Entropy?

In the 1800s, physicists developed Laws of Thermodynamics to help them figure out how to build more efficient heat engines. A heat engine is a machine that converts heat into useful work. An example would be a steam engine that burns coal and uses steam to push pistons to turn the wheels of a train to move it along the tracks. A refrigerator is a heat engine run in reverse: instead of putting heat in to get work out, it puts work in to get heat out. In both cases, the efficiency is less than perfect: there is always some wastage. Engine efficiency is imperfect even for the best possible heat engine/fridge design we can theoretically imagine, which runs on what is known as the Carnot cycle.

Thermodynamics is a general set of ideas allowing physicists to describe macroscopic flows of thermal energy, heat, and work for any given substance containing zillions of tiny molecular/atomic constituents in terms of just a few gross properties of the substance like the temperature and the pressure. The entropy is one of those gross properties, and it encodes how much heat is wasted while a heat engine does work (or while a refrigerator extracts heat). Entropy can alternatively be thought of as describing the degree of disorganization of a system. For example: if my desk is tidy, that has low entropy; if my desk is messy, that has high entropy. Another example: ice has lower entropy while liquid water has higher entropy, because the molecules in [ordinary] ice are constrained in a tidy hexagonal crystal structure, while the molecules in liquid water are free to move about independently of one another in a messier fashion.

Laws of Thermodynamics

Zeroth Law

This one sometimes feels too basic to even state and so gets left off the list. It concerns systems that have reached thermal equilibrium, which is what happens when enough time has elapsed that there is no longer any net flow of energy. In a nutshell, this law is about relationships between systems. The Zeroth Law of Thermodynamics says:

0th Law: if system A and system B are in thermal equilibrium, and system B and system C are also in thermal equilibrium, then system A and system C must be in thermal equilibrium too.

First Law

The First Law of Thermodynamics is basically the principle of energy conservation applied to thermodynamic processes. It says:

1st Law: during a thermodynamic process, the change in the thermal energy of a system is the sum of the heat added to it and the work done on it.

Both the heat transferred in and the work done typically depend on the path, i.e., on details of how you perform the thermodynamic process. What is not obvious at first glance -- and really cool and useful -- is that the total thermal energy of a system is only a function of the current state of the system, not how it got there.

Second Law

A reversible process in physics is one in which you can wind the clock backwards to get both the system and its environment back into their original states. In the context of thermodynamics, reversibility requires that the process be done slowly enough that you can maintain thermal equilibrium throughout (to a very high degree of accuracy). By contrast, sudden processes are irreversible.

For all the most common molecular/atomic processes, running the clock backwards makes as much sense as running it forwards. So why is this microscopic story no longer true when you are dealing with a macroscopic system? After all, we intuitively know when a movie involving macroscopic human-sized things is being run in reverse. Coffee and added milk don't spontaneously un-mix, a delicate vase knocked to the floor and smashed into pieces doesn't spontaneously re-constitute and jump back up onto the shelf, and heat doesn't spontaneously travel from a colder body to a hotter body. Physicists codified these observations and a wide range of others into a general principle, by making use the of the idea of entropy for a closed system, which is a system for which no net energy or particles go in or out. This principle is known as the Second Law of Thermodynamics:

2nd Law: the entropy of a closed system does not spontaneously decrease over time.

So how is it that Earth, over 4.5 billion years, has developed more complicated life-forms and become more organized in important ways? Doesn't that violate the Second Law? No, because Earth is actually not a closed system! It receives lots of continuous energy input from the Sun, and radiates some energy out, and also exchanges particles with outer space. So Earth's entropy can decrease, as long as the entropy of our Solar System surroundings increases at least as much, which it does. How about the universe? If we mean the whole shebang, then yes, it is by definition a closed system. Incidentally, the reason why the arrow of time points in the direction it does has to do with the details of how our universe was born.

So why is the Second Law of Thermodynamics true? In essence: reversible microscopic processes lead to irreversible macroscopic behaviours because some of the possible states of the macroscopic system are vastly more probable than others. For macroscopic systems, which are composed of a huge number of microscopic ingredients, only the very most probable thing is likely to happen. In other words, what underlies the Second Law is simply the statistics of large numbers. To give a rather silly example: it is in principle possible for all the air molecules in Ontario up to fifty stories high to spontaneously gather in the palm of my hand right now, reducing entropy and suffocating millions, but it is so improbable compared to the normal situation that on average it wouldn't be expected to happen even once in the current lifetime of the universe. Whew!!

If you would like to see a simple example of the math underlying the statistics of large numbers, scroll down to the Appendix at the bottom of this page.

Third Law

Is there a Third Law of Thermodynamics? Yes. It is based in observations that the colder something gets, the harder it becomes to cool it down further. There are multiple ways of stating this law, but for our purposes this one is easiest to grasp:

3rd Law: it is impossible to cool a system all the way down to absolute zero in a finite time.

As in life, some things in physics are inherently harder to do than others.

The 3rd Law implies something interesting about heat engines (and refrigerators): that it is impossible to build a 100% efficient one. The Carnot cycle is the most perfect heat engine (or refrigerator) cycle imaginable, and even it necessarily has less than 100% efficiency -- unless it can access a thermal reservoir at a temperature of either (a) absolute zero or (b) infinity. Neither of these theoretical options is physically realistic: (a) is impossible by the Third Law, while (b) would require an infinite amount of energy to make and maintain, and even the entire universe taken together doesn't have that much energy!

So how about purported perpetual motion machines, which supposedly let you get something for nothing, say via a heat engine or fridge with greater than 100% efficiency? The instinctive reaction I would have to someone who claims they can make one would be to laugh. Or if I were in a kinder mood, I would suppress my smile and patiently explain that what they are trying to do is literally completely impossible. No matter how imaginative they might think they are, they can't beat the simple statistics of large numbers. Many have tried, all have failed, and all who try in future will fail too. There are better problems to be working on, like ending homelessness or averting catastrophic climate change.

Appendix

Consider a simple thought experiment where we flip some number of evenly weighted coins, each of which has heads (H) on one side and tails (T) on the other. What possible outcomes can we get? We can enumerate them. Let us define the microstate to describe the precise state of the individual coins after they are flipped, the macrostate to describe the total count of heads vs tails, the number to be how many different microstates can give that microstate, and the probability to be how likely that macrostate is out of all the possible options.

For one coin:

For two coins:

For three coins:

For four coins:

As you keep going, it rapidly becomes more laborious! For eight coins:

And so forth. For any number of coins, the average behaviour is always half H and half T. But as we increase the number of coins, the probabilities concentrate closer and closer to the average. One way to see this fact is to look at the all-H probability or the all-T probability for each case listed above: they get a lot smaller as the number of coins grows. In statistics lingo: as the number of coins becomes very large, the distribution of probabilities approximates a bell curve, with a width that becomes extremely small as a fraction of the average.

Just how large of an number of microscopic ingredients are we typically dealing with, for macroscopic quantities of stuff we might study in the lab? Consider one mole of oxygen gas, which weighs 32.0 grams and at zero degrees Celsius and atmospheric pressure takes up about 22.4 litres of space. This contains about 602,200,000,000,000,000,000,000 oxygen molecules! In other words: for macroscopic amounts of stuff, the number of microscopic constituents is staggeringly enormous, so essentially only the very most probable thing happens.

---

2025-03-08