Nevit Dilmen M.D., March 2012, Istanbul
Last rev. May 2016, İstanbul
No conflict of interest disclosures
Revisions Başak Torum Eroğlu, M.D., Jan 2016, İstanbul
In this study, the methods which I use to create full-color trichromatic MRI images and their implications are described. Color MRI images can be valuable clinically and their potential is uninvestigated and underutilized. Three gray-scale images are converted to three respective channels in displayed RGB screens. The resulting images are false color since they do not match tissue color, but can be clinically interpreted if resulting color information is properly understood and discussed. The method has no standard algorithms such as automatic segmentation used in previous attempts for colorizing MRI. Further research is necessary to prove and clinically utilize the method.
Mechanism of trichromatic color vision
Trichromacy or tri chromaticism is the condition of possessing three independent channels for conveying color information, derived from the three different cone types. A true-color image of a subject is an image that appears to the human eye just like the original subject would: a green tree appears green in the image, a red apple red, a blue sky blue, etc. When applied to black-and-white images, true-color means that the perceived lightness of a subject is preserved in its depiction. In a false-color image set, human visual perception is limited to three independent coordinates (combination of red, green, and blue); at most, three measurements can be depicted in such a way. A pseudo-color image is derived from a greyscale image by mapping each pixel value to a color according to a table or function. Primates, that is, monkeys, apes, and man, are the only known mammalian trichromats. Each of the three types of cones in the retina of the eye contains a different type of photosensitive pigment called opsin. Each different pigment is especially sensitive to a certain wavelength of light approximated to red green and blue ≈(420, 530, 560 nm) wavelengths. There is another photosensitive cell called rod which predominantly functions in sensing the intensity of light. Tones of gray stimulate three 3 cones in an equal manner. They also stimulate rods which are sensitive to the intensity of light in a wider wave length. By displaying the images in gray-scale we predominantly utilize rods in sensing the information and under-utilize cones in the retina. This is similar to pre-color television era.
Image: Nevit Dilmen, http://bit.ly/Tricolor1
The RGB color model is an additive color model in which red, green, and blue lights are added together in various ways to reproduce a broad array of colors. A typical color digital image contains three channels for red, green and blue.
Combination of three grayscale images in different orders results in different colors Image: Nevit Dilmen http://bit.ly/RGBComp
Digital color images are often built of three stacked color channels, each of them representing value levels of the given channel representing red, green and blue. The image can be decomposed to three grayscale images that can be recomposed back to RGB. To regain the original colors during re-composition the order of Red green and blue channels should be retained. RGB > RGB. If the order changes the colors will look different since they stimulate red, green and blue in different ratios than original colors. If we assign letters A, B and C to initial 3 gray-scale images, there would be six possible RGB combinations available for each set of 3 images. If A is assigned to Red, B to Green, and C to Blue = we can show this order as ABC. The six combinations would be ABC, ACB, BAC, BCA, CAB, CBA.
When an area is bright on any of color channels and dark on others the pixel on that location shows a vivid color in green blue or red. When an area is bright on two of color channels and dark on other the pixel on that location shows colors cyan (C), yellow (Y) and magenta (M). Y=R+G, C=G+B, M=R+B. When an area is bright on three of color channels the pixel on that location shows as white. When an area is dark on three of color channels the pixel on that location shows as black. Without considering the difference in sensitivity of our eyes to red, green and blue colors, when a single color is at maximum intensity and the others at a minimum, the brightness of a pixel is at ⅓ of maximum. When two colors are at maximum intensity and the other at a minimum, the brightness of a pixel is at ⅔ of maximum. So resulting color vision has a price ; when we increase color resolution, we are actually decreasing brightness resolution.
Close-up image of mouse arrow on a computer screen, showing color pixels. Image: Nevit Dilmen http://bit.ly/Arrow1
When the image is displayed on the screen, three different intensities are applied to each pixel of the display which corresponds to a value of red, green and blue channels.
One of the earliest tricolor images. by Sergei Mikhailovich Prokudin-Gorskii Public domain (old) http://bit.ly/AlimKhan
A picture of Alim Khan (1880-1944), Emir of Bukhara, taken in 1911. This is an early color photograph taken by Sergei Mikhailovich Prokudin-Gorskii as part of his work to document the Russian Empire. Three black-and-white photographs were taken through red, green and blue filters. The three resulting images were projected back through similar filters. Combined on the projection screen, they created a full-color image. The negative sets were scanned and digitally combined by Library of Congress to get the above image.
Additional known methods of creating multichannel color images from multiple gray-scale images (HSV, HSL, CMY, CMYK, Lab) are not discussed here.
Materials & Method
The images used for this study are acquired with 1.5T Siemens Symphony Vision in a private radiology center. Grayscale images were snapshots from PACS station. GIMP (gnu image manipulation program), a freely available and open source image manipulation program was used to compose images to RGB.
Application to MRI images
Displayed gray-scale MRI images are composed of shades of gray, varying from black at the weakest intensity to white at the strongest. The intensity of brightness in these images corresponds to signal intensity of tissue in corresponding imaging sequence. Each sequence is unique since they display different tissues as bright and dark areas.
Brain MRI patient with Lung cancer metastasis. T1 corresponds toRed, PD assigned to Green and T2 to Blue. Image: Nevit Dilmen
I assigned 3 MRI images acquired with different sequences to 3 different channels of red green and blue to generate a color image. The colors in the images are false since they do not correspond to the color of the normal tissue. Technically these are true colors since they have 24-bit color information. They are not color map images, which are frequently used to create a different type of false-color images. In color mapping with relatively low color depth, the stored value is typically a number representing the index into a color map or palette. In typical 8bit files, color maps can display a maximum of 256 colors, each color corresponding to one shade of gray. To avoid confusion I suggest using trichromatic MRI or tricolor MRI instead of false color or true color designations. Trichromatic (3 channel) color images can display a maximum of 16.7 million colors. So a trichromatic, 24bit MRI image reveal 65536 times more information than a single channel grayscale image or any tone mapped color image.
Which pictures to combine and in what order?
Each set would have it’s own advantages and disadvantages. Two kinds of considerations might be used. A. Ease of psychological comprehension and interpretation with an easy to learn learning curve B. Improved diagnostic accuracy with possible steep learning curves. I do not have the resources to test diagnostic specificity, sensitivity and diagnostic accuracy of hundreds of scenarios. The area is open to further research.
Theoretically, any three image set can be combined and they can be combined in any order. However to have meaningful information in images and ease of communication between medical doctors combinations should be defined for each region. One of the first possibilities is to assign A: T1, B: PD, C: T2. There is no limitation of possible sequences to explore. T1, PD, T2 are given as example since they are some of the oldest and most commonly used sequences. Other possibilities to explore in channels are diffusion images with X,Y,Z orientation info, Diffusion, Trace, and ADC, fat saturated and water saturated images.
One option is to make images that display water in shades of blue. The main feature of combinations which display water in shades of cyan blue should be high signal intensity in blue or both blue and green channels. For brain images, this means T1 in the red channel, PD in green and T2 in the blue channel. Ventricles, cisterns, orbits, sub-arachnoid space appear in a cyan / blue tone with such a composition. This combination showed fat in yellow/white. With this combination gray matter appeared in a greenish tone while the white matter was displayed in brownish red, muscles were dark green colors.
In tumors a T1 contrast-enhanced (CE) image can replace T1 for comparison. By using the same convention we can add a 4th letter D for contrast enhance T1 images. Typically the colors in DBC, DCB, BDC, BCD, CDB, CBD would be similar to ABC, ACB, BAC, BCA, CAB, CBA respectively. We can see that the contrast enhancement of metastasis, in this case, is more obvious in some color combinations than others. It is more obvious when D is either Red or Green and less obvious when D is Blue. It can be due to the fact that our cones are less sensitive to blue than to red or green.
Brain MRI of a patient with metastatic lung cancer. A, B, C: Source images T1, PD, and T2 respectively. D: Contrast enhanced T1. Combined to RGB in different orders. Metastasis is more readily seen when T2 is on Green channel. In contrast enhanced images effect of contrast is minimal when CE-T1 is on Blue channel. Image: Nevit Dilmen
When there are only two images available, it would be possible to combine them to create a dichromatic MRI image. One of the images would be in two channels or one of the channels would be blank. Possible combinations are: AAB, ABA, ABB, AxB, ABx, BAA, BAB, BBA, BxA, BAx, xAB, xBA. ( x used for the blank image.) See bichromatic image of the cervical spine in patient with syringomyelia. T1 assigned to red, T2 to green and blue.
How to read the images
The color images will display colors according to the image source images used in their creation. If the pixel has low values in all three channels the resultant color would be black. If all three channels have high signal intensity the pixel will glow bright white. The high signal intensity in red and green channels with low signal in blue will result in yellow color. The colors in RGB combinations are well known: R+G+B = White, R+G= Yellow, R+B = Magenta, G+B = Cyan
Consistency among images
Considering that the same sequence of combination used for converting to RGB, relative visual consistency among images can be achieved. Below are different images respectively T1, PD, T2, with manually modified window levels and centers. In all images, CSF remains blue-cyan whatever the tone, gray matter is always green, white matter reddish-brown. The tonal differences among images which can be either attributed to image acquisition parameters or windowing center and levels when converting acquired MRI to 8bit grayscale images. Further studies are needed to workout level of standardization needed for clinical practice.
Color MRI image acquisition should be performed so that each pixel at same (x,y) of three channels represent the same voxel of the patient. The technician should not modify zoom and rotation factors during acquisition. Patients should avoid moving and rotating between data acquisitions. The same principle will be applied if one of the sequences are going to be contrast enhanced. Preparations should be made so that there would be no need to move patient during contrast agent administration. Movements between acquisitions would result in color misregistrations and artifacts. Sometimes such as in the case of bowel movement organ motion might be unavoidable or difficult to prevent. Color MRI might be unsuitable for such examinations.
Image acquisition for colors in all three planes would mean long examinations. Since each color image needs 3 sequences.
Automation of procedure
Manual aligning is time-consuming and laborious. Software generation of tricolor images should be used to study series of images and start of large-scale studies. multiple radiologists who are familiar to color images should encounter for comparison of diagnostic accuracy and possible advantages. I tried to describe some color generation principles in order to ease comprehension of color images.
Alternative naming for trichromatic MRI can be proposed. Three color MRI, three channel MRI, 16M color MRI, RGB MRI are only a few.
Possible advantages of trichromatic MRI
Ease of application:
Color MRI can be added to the software of most MRI machines at software programming cost. With available technology, it would be possible to produce color images from existing hardware and protocols. Color composing feature can be added to image viewing workstations. The workstation would receive grayscale images and produce trichromatic images on demand. With software support, it would be possible to use four sliders to adjust window width and center in each of red, green, blue channels and all RGB at once.
Coloring the images increases color resolution from 256 gray tones to 16.7 Million colors. By agreeing on certain sequence/channel combinations or protocols it would be possible to associate certain colors with structures and lesions. Call meningioma as green, CSF, edema as blue, white matter as brown/red, lipid as yellow which may ease learning curves and remember lesions. Radiologists valuable time can be regained if radiology teams could go only through one color series of color images during report generation process instead of three sets of gray-scale images. Gray-scale images would always be available as a reference and could be consulted during the transition period and in the case of any doubt.
Possible disadvantages of Trichromatic MRI
Despite the increase in color resolution and information, a certain decrease in brightness resolution will be unavoidable. Decrease in brightness resolution means a decrease in the number of lightness levels eye can perceive. Digitally converting a color image to a representative gray-scale can be achieved with different formulas. Luminosity formula attempts to take perceived strengths of Red Green and Blue. Average is straightforward mean of red green and blue. Each has its advantage but as you can see all have lower contrast than the original image. The lightness method averages the most prominent and least prominent colors.
Lightness: The gray level will be calculated as Lightness = ½ × (max(R,G,B) + min(R,G,B))
Luminosity: The gray level will be calculated as Luminosity = 0.21 × R + 0.72 × G + 0.07 × B
Average: The gray level will be calculated as Average Brightness = (R + G + B) ÷ 3
This can be avoided by consulting original gray-scale images, at a cost of increased time of examination. Any image assigned to the blue channel will be in a disadvantageous position since our eyes are less sensitive to blue than green and red. Obviously, all three gray-scale source images should be easily available for interpreting in the case of any doubt.
Until now radiologists was not tested for color blindness. Radiology was a B&W profession. But utilizing color MRI may need for a color test before a radiologist can be certified to interpret clinically relevant information in color.
Although viewing images from high-resolution color monitors should not place any problems on trichromatic MRI. Changes would be necessary for printing color MRI’s on transparent films. Another possibility is front lighted color paper printing. Possible information loss during printing should be assessed separately since most printing systems involve converting RGB image to CMYK which is a process with some information loss. It should be assessed that whether this RGB>CMYK conversion results in loss of clinically significant information. The lost colors are called out of Gamut colors.
The lower sensitivity of our eyes to blue color might need special attention and work up.
Radiology community might have to re-learn in color what has already learned. Out of standard color images and lots of presentation, possibilities may lead to confusion and difficulties in interpreting the results.
A huge potential is described above and should be further investigated to bring light to the full potential.
False-Color Multi-parameter MRI Visualization
Nature Hot Topic, Color MRI on the horizon, June 19, 2008
An Attempt to Develop Color Magnetic Resonance Imaging (MRI) by using Visible Human Data, Shigeru Muraki, Toshiharu Nakai and Yasuyo Kita,
A NEW METHOD FOR COLOURIZING OF MULTICHANNEL MR IMAGES BASED ON REAL COLOUR OF HUMAN BRAIN, 15th European Signal Processing Conference (EUSIPCO 2007), Poznan, Poland, September 3-7, 2007 Mohammad Hossein Kadbi; Emadoddin Fatemizaheh; Abouzar Eslami; and Maryam Khosroshahi
H. Keith Brown has developed technology that creates color fusion images by software interpretation of the multi parameter MRI images.
Transferring Color to Greyscale Images, Tomihisa Welsh, Michael Ashikhmin, Klaus Mueller
G. N. Holland and P. A. Botomley, “A Color DisplayTechnique for NMR Imaging,”Journal of Physics E:Scientific Instruments, Vol. 10, No. 7, 1977, pp. 14-16.doi:10.1088/0022-3735/10/7/014
M. C. Valdes Hernandez, K. J.Ferguson, F. Chapell and J.M. Wardlaw, “New Multispectral Data Fusion Techniquein MRI for White Matter Lesion Segmentation: Methodand Comparison with Thresholding in FLAIR Images,”European Radiology, Vol. 20, No. 7, 2010, pp. 1684-1891.doi:10.1007/s00330-010-1718-6
W. E. Phillips, H. K. Brown,J. Bouza and R. E. Figueroa,“Neuroradiologic MR Applications with MultiparametricColor Composite Display,”Magnetic Resonance Imaging,Vol. 14, No. 1, 1996, pp. 59-72.doi:10.1016/0730-725X(95)02043-S
H. K. Brown, T. R. Hazelton and M. L. Silbiger, “Gen-eration of Color Composites for Enhanced Tissue Differ-entiation in Magnetic Resonance Imaging of the Brain,”American Journal of Anatomy, Vol. 192, No. 1, 1991, pp.23-34.doi:10.1002/aja.1001920104
H. K. Brown, T. R. Hazelton, J. V. Fiorica, A. K. Parsons, L. P. Clarke and M. L. Silbiger, “Composite and Classi-fied Color Display in MR Imaging of the Female Pelvis,”Magnetic Resonance Imaging, Vol. 10, No. 1, 1992, pp.143-154.doi:10.1016/0730-725X(92)90384-C
M. G. Wells, P. F. Sharp andA. N. Law, “Principles and Appraisal of Combined Images in NMR,”Medical andBiological Engineering and Computing, Vol. 27, No. 3,1989, pp. 277-280.doi:10.1007/BF02441485
R. L. Kamman, G.P. Stomp and H. J. Berendsen, “Uni-fied Multiple-Feature Color Display for MR Images,”Magnetic Resonance in Medicine, Vol. 9, No. 2, 1989, pp.53-55
Some of the images used in this document and more trichromatic MRI images
More Images on Nevit Dilmen’s blog
Converting color to Grey scale, Desaturation
Composting 3 gray scale images to RGB
Motion artifacts in trichromatic abdominal MRI
Link to this page: