A clean beam suddenly starts to flicker
Around 2010, an odd sight was reported from a high-power, rod-type fiber laser system. Its output had been a smooth Gaussian beam. Then, once the average power reached a certain level, the beam suddenly broke apart and began to flicker. The report of this observation started research on the effect around the world.
The beam's intensity pattern never settled down over time. That behavior pointed to higher-order modes appearing at the fiber output, so the effect was named transverse mode instability (TMI).
A transverse mode is a shape that light settles into across the fiber's cross section. The simplest, single-lobed shape is the fundamental mode, and the more complex shapes are higher-order modes. A single-mode fiber is a fiber designed to guide only the fundamental mode.
Readers of the earlier articles may find this a little puzzling. "What is a fiber laser? Growing light inside a thin glass fiber" said that the fiber's long, thin shape gives "considerable resilience" to the effects of heat generated in the core. Resilience does not mean there is no heat problem.
The abstract of a 2010 review paper also named the key factors limiting fiber laser performance: thermal management, optical nonlinearity, and damage.
The previous article covered nonlinear effects such as stimulated Brillouin scattering (SBS) and stimulated Raman scattering (SRS). This one covers heat. It follows where heat comes from, how it changes the glass, and what marks it leaves on the beam. It then explains how TMI happens and why it has been called the biggest limit on raising average power.
Where the heat comes from: quantum defect and other sources
The light of a fiber laser comes from pump light. Ions added to the core absorb pump photons and then give off signal photons, often with a longer wavelength. A pump photon usually carries more energy than a signal photon. The energy difference between the two photons is called the quantum defect, and that leftover energy is lost as heat.
A fiber with light-amplifying ions in it is called an active fiber. An active fiber has several heat sources. A 2020 review paper on TMI names the quantum defect as the most important one. This heat is built into laser operation and comes from the very process that makes signal photons, so the review calls it "productive" heat.
Some numbers give a sense of scale. A 2026 review chapter on single-frequency fiber amplifiers, posted as a preprint on arXiv, gives an example. An ytterbium-doped fiber (YDF) amplifier with a 1064 nm signal is commonly pumped at 915, 940, and 976 nm for good absorption. Photon energy is inversely proportional to wavelength, so the share of one pump photon's energy that does not become a signal photon can be found like this.
Quantum defect share = 1 − pump wavelength ÷ signal wavelength
Working it out gives about 8.3% for a 976 nm pump, about 11.7% for 940 nm, and about 14.0% for 915 nm. These are ideal ratios for one pump photon turning into one signal photon. They are not the actual heat output or efficiency of any laser. Pump light that passes through without being absorbed, and the other heat sources below, are not in this count.
This heating shrinks as the pump wavelength moves closer to the signal wavelength. That is why "Double-clad fibers and how pump light gets in" said that moving the pump wavelength toward the signal aims to cut heating from the quantum defect.
The quantum defect is not the only heat source. There are also "non-productive" sources, where photons vanish and become heat without making any light. One of them, linear ground absorption, is extremely low in most situations today. Its share of the total heat load is usually negligible.
In ytterbium-doped fibers, photodarkening deserves more attention. It is a gradual loss of the fiber's transparency as the laser runs longer. It is widely accepted that the added absorption comes from color centers forming in the fiber, but exactly how those color centers form is not yet fully understood. The energy they absorb turns into heat and adds to the fiber's total heat load.
Before TMI research began, photodarkening was thought to be nearly solved, or at least under control, for most practical uses. TMI research changed that view. It showed that even a low level of photodarkening can strongly affect the TMI threshold through the extra heat it adds.
Heat changes the glass. In silica glass, the refractive index rises with temperature. The refractive index describes how slowly light travels through the glass, and a fiber traps light by making the core's index slightly higher than that of the surrounding cladding.
Fibers handle heat well, but the heat is still there
Heat has long been a problem for lasers. In early lasers, raising the average power distorted the output beam and sharply reduced its spatial coherence, meaning how well points across the beam oscillate in step. To solve this, designers changed the shape of the active medium to give it more surface area for its volume. Slabs, thin disks, and fibers are examples, and a larger surface area makes it easier to carry heat away.
Fibers stand out for more than their long, thin shape. Among high-power active media, only fibers have a waveguide structure that traps and guides light. Thanks to this, fiber lasers can deliver diffraction-limited beams over a wide range of average power, even under a high heat load. A diffraction-limited beam is one with the best beam quality, spreading only as much as physics allows.
Fibers achieved this despite a weakness of glass. The 2020 TMI review points out that glass has a thermal conductivity roughly 10 times lower than the crystals used in other laser media. Even so, when the review was written, fibers held the record for the highest average power with diffraction-limited beam quality: 6 kW from an ytterbium-doped silica fiber.
There was a cost. As their signal cores grew larger, active fibers for high power became steadily shorter, and the heat load rose sharply as output climbed. As a result, thermal effects began to appear in active fibers.
Large cores have another problem. A large signal core can usually guide light in several transverse modes, which hurts beam quality and stability.
Even a single-mode fiber, built to guide only the fundamental mode, can still carry higher-order modes. They simply remain as high-loss modes that fade quickly over short distances. Such modes can be excited by flaws in the fiber, bending, imperfect input coupling, or index changes caused by heat.
Heat even reduces that loss. When the core heats up more than the cladding, their index difference grows. So does the numerical aperture (NA), a measure of the range of angles at which light is accepted. Higher-order modes then leak away less, and a single-mode fiber gradually turns multimode as power rises.
An ultrafast laser textbook shows the same concern from another side. Ytterbium-doped lasers can be pumped with a very low quantum defect, which has greatly raised the average power of ultrafast systems. In such systems running at several hundred watts of average power, though, the key is removing heat without adding thermo-optic distortions. Saying fibers handle heat well means they have a shape that makes heat easier to manage, not that there is no heat.
Two marks heat leaves on the beam
The 2020 TMI review sums up thermal effects in fibers as having two main forms. One is the shrinking of the mode size, and the other is TMI.
Mode shrinking means the mode field diameter gets steadily smaller as the average power rises. The mode field diameter is the width of the cross section that the light spreads across. As the previous article showed, nonlinear effects arise more easily when light is squeezed into a small space, so a shrinking mode makes them stronger. It partly cancels the benefit of enlarging the core to avoid scattering.
TMI has more dramatic results. Once a certain average power is reached, the quality and stability of the beam from a fiber laser suddenly drop. The review chapter on single-frequency amplifiers defines TMI as a thermo-optically induced dynamic coupling between the fundamental mode and higher-order transverse modes. This matches the wording in "Double-clad fibers and how pump light gets in": "a dynamic coupling between the fundamental mode and higher-order transverse modes, driven by thermo-optic effects."
The thermo-optic effect is the way temperature changes the refractive index. A 2024 theory paper on TMI describes the same effect as a dynamic transfer of power between the fiber's transverse modes. It is caused by thermo-optical scattering, by fluctuations in the population inversion, or by both. Whether it is called "coupling" or "power transfer," light energy moves back and forth between modes and the beam's shape changes.
How transverse mode instability was first seen
Even within the 2020 TMI review, the year of the first observation is given slightly differently: 2009/2010 in the section on experiments, and 2010 in the conclusion. The scene is the same in both. In a high-power, rod-type fiber laser system, the Gaussian beam suddenly broke apart and began to flicker at a certain average power.
Seeing the flicker properly took a fast camera. The fluctuations were much faster than the 25 Hz frame rate of ordinary cameras. High-speed camera footage revealed three things.
- The beam fluctuations happen on a millisecond (ms) time scale.
- The beam is always a coherent overlap of two or more transverse modes.
- Between a still beam and a wildly fluctuating one, there is a transition region where the beam changes periodically.
When the mode content was worked out for each frame of the footage, it confirmed that energy really does move between different transverse modes once TMI begins. An almost complete transfer of energy between the fundamental mode and a higher-order mode can happen on a millisecond time scale.
Another clue was that the fluctuations began suddenly at a certain average-power threshold. This suggested that TMI, unlike the classical nonlinear effects already known in fibers, depends not on the light's peak power but on its average power.
Later, photodiode measurements laid out three operating states that depend on average power. In the stable state below the threshold, the beam stays still. In the transition state slightly above the threshold, the beam changes periodically, and in the chaotic state well above it, the beam's shape changes in complex ways.
Here "chaotic" does not mean chaos in the mathematical sense. It is a loose word for fluctuations that cannot be predicted. The further the power goes above the threshold, the more modes join the fluctuations and the faster they become. As a result, at high enough power, the time-averaged beam approaches a flat-top shape.
Evidence that heat is the cause also built up. In fibers with different core sizes, the larger the core, the slower the beam fluctuations. Their main frequency matched the time it takes heat to spread through the core. This result strongly suggested a thermal origin for TMI.
In experiments with a pulsed pump, the fluctuations also took several ms to build up, which reinforced that suspicion.
TMI is not tied to one kind of ion either. Most studies used ytterbium-doped fibers, but TMI was also predicted in thulium-doped and Raman fiber amplifiers. By the time the 2020 review was written, TMI had actually been observed in thulium-doped amplifiers, which proved to withstand it far better than ytterbium-doped ones.
From this, the 2020 review concludes that TMI is a fundamental thermo-optical effect not tied to any particular laser-active ion. In any amplifier, TMI can be expected once a certain threshold heat load is reached.
A grating made of heat: how TMI happens
One device that moves energy between two modes inside a fiber is the long-period grating. It is a structure whose refractive index changes in a repeating pattern, with a period much longer than the signal wavelength. In TMI, heat made by the light itself plays the role of such a grating, and the process can be pictured in four steps.
1. Two modes overlap. The fundamental mode and a weak higher-order mode travel along the fiber at different speeds, called phase velocities. Their combined light therefore forms an interference pattern along the length, where the modes add up into bright spots and partly cancel into dark ones. Even if the higher-order mode is so weak that it seems negligible, a shallow intensity pattern still forms.
2. Heat pattern. Where the light is strong, the signal uses up more of the population inversion, so the intensity pattern is imprinted on the inversion profile. Heat sources such as the quantum defect then release heat following this pattern, in proportion to the local light intensity. Brighter spots get hotter, creating a temperature pattern.
3. Refractive-index grating. Silica's refractive index rises as it gets hotter, so the temperature pattern becomes a refractive-index pattern. This is a long-period grating made by heat. Because it is born from the interference pattern, it always has the right period and symmetry to move energy between those two modes. Building on this heat-made grating, a widely held view treats TMI as a form of stimulated thermal Rayleigh scattering (STRS) in fibers.
4. Energy moving between modes. At low power, the heating is weak, the grating barely scatters light, and the amplifier stays stable. As the power rises, the temperature contrast grows and the grating gets stronger. If the grating sits slightly offset from the interference pattern, energy moves from one mode to the other. The intensity pattern then changes, and so does where the heat is made, closing a feedback loop back to step 1.
Picture wind shaping ripples in sand, and the ripples in turn changing how the wind flows. The real thing differs in an important way. In TMI, the pattern is not sand but temperature and refractive index shaped by the light. What it changes in return is the light's own mix of modes.
The offset in step 4 is the second condition for energy transfer. A grating alone does not make energy move; the interference pattern and the refractive-index grating must be slightly offset along the length of the fiber. This offset is called a phase shift, and the TMI threshold is set by the combined strength of the grating and the phase shift.
Where the phase shift comes from is still under discussion and has not been fully explained. The 2020 review presents two hypotheses and says its authors support one of them, and it notes at the start that it is written from the authors' point of view.
The instability comes from heat being slower than light. Light responds almost instantly to a change in refractive index, but the temperature pattern follows more slowly because heat takes time to spread. This delay makes power slosh back and forth between the modes on a millisecond scale. It is the same time scale as the fluctuations caught by high-speed cameras.
Much like SBS, this thermo-optical scattering acts as a noise amplifier. Small fluctuations over time in the seed light or pump power can grow through the thermo-optical feedback. At high power, even a very small amount of higher-order-mode content is enough to start the instability.
The "threshold" and the average-power limit
The TMI threshold is an average-power limit that differs from system to system. When the 2020 review was written, it usually fell anywhere between 100 W and several kilowatts, depending on the fiber laser system. The 2024 theory paper also defines the TMI threshold as the output power above which beam quality degrades significantly.
There is more than one way to set the threshold. A common method raises the power while measuring the standard deviation of the beam fluctuations, then fits a function to the rising curve to set the threshold as a number. The 2020 review points out that this definition is widely used but is not the only one. In systems where the transition is fairly abrupt, the threshold is sometimes simply taken as the power where the beam's stability changes in a step.
Some studies define the threshold using M², a number that describes beam quality. "Beam quality: M², divergence and brightness" explains M² in detail. M² can get worse in a fixed, static way that has nothing to do with TMI, though. So a definition based on M² can be less reliable than the ones above.
So what moves the threshold? The 2020 review holds that because TMI comes from heat, possibly the most important factor is the heat load in the fiber. The review chapter adds the noise sources, their spectral features and size, and the amplifier design. Photodarkening, mentioned earlier, can also shift the threshold a lot even in small amounts.
Depending on average power sets TMI far apart from the scattering in the previous article. Raman and Brillouin scattering and self-phase modulation (SPM) depend on the light's intensity in the core, so enlarging the core to lower intensity helps. That approach works only on intensity-dependent effects. TMI is a thermal effect that depends on average power, so the hard-won knowledge about conventional nonlinear effects cannot simply be carried over.
The conclusion of the 2020 review calls TMI the first example of a heat-induced nonlinear effect in optical fibers. After it was first reported in 2010, it quickly became the strongest limit on raising the average power of fiber laser systems. When the review was written, it was still the strongest limit.
TMI also shook the fiber laser's reputation for beam quality that does not depend on power. When the review was written, there was a spreading fear that TMI may be the ultimate limit on fiber laser average power. In the opening sentence of a 2024 theory paper's abstract, TMI is again named the primary limit on power scaling in high-power fiber lasers and amplifiers.
Narrowing the view to single-frequency fiber amplifiers, which boost light with a very narrow linewidth, the nonlinear effects with the lowest thresholds are SBS and TMI. SBS was covered in the previous article, "Limits of a single fiber (1)."
What to read next
- "Beam quality: M², divergence and brightness": shows how the "diffraction-limited beam" and "beam quality" mentioned here are measured with M² and divergence.
- "Why beam combining is needed, and its three approaches": looks at why and how beams from several fibers are brought together into one.