The independent component analysis method was used to study the defect characterization problem of composite impact damage based on pulsed infrared images. First, according to the prior knowledge of pulse infrared experiment of test block and the temperature difference between defect and non-defect, the appropriate interval segment was selected for infrared sequence image. Then, centering preprocessing and whitening preprocessing were performed on the constructed data matrix, independent component analysis was performed by gradient descent method, and the independent component feature image can be obtained by reconstructing the matrix. Finally, by choosing an appropriate image segmentation method, the defect characterization quantities of the maximum damage long diameter and damage short diameter can be obtained. Taking the damage defect extraction of two carbon fiber reinforced composite laminates with different thicknesses under the impact energy of 30 J and 40 J, as an example, the maximum damage long diameter and damage short diameter of impact damage were successfully extracted and compared with the results of ultrasonic F-scan.  For comparison, the errors are within an acceptable range, especially for the laminate with a thickness of 2 mm, which errors are within 5%. The research results showed that independent component analysis can effectively distinguish noise from defects, and the obtained feature image has higher signal-to-noise ratio, greater contrast and better image quality than the original image, which is beneficial to the extraction and characterization of defects.