Challenging illumination conditions (low-light, under-exposure andover-exposure) in the real world not only cast an unpleasant visual appearancebut also taint the computer vision tasks. After camera captures the raw-RGBdata, it renders standard sRGB images with image signal processor (ISP). Bydecomposing ISP pipeline into local and global image components, we propose alightweight fast Illumination Adaptive Transformer (IAT) to restore the normallit sRGB image from either low-light or under/over-exposure conditions.Specifically, IAT uses attention queries to represent and adjust theISP-related parameters such as colour correction, gamma correction. With only~90k parameters and ~0.004s processing speed, our IAT consistently achievessuperior performance over SOTA on the current benchmark low-light enhancementand exposure correction datasets. Competitive experimental performance alsodemonstrates that our IAT significantly enhances object detection and semanticsegmentation tasks under various light conditions. Training code and pretrainedmodel is available athttps://github.com/cuiziteng/Illumination-Adaptive-Transformer.