The energy conservation equation can be written as,

The expression relating thermal energy density and temperature u(x, t) is,

where c(x) is the specific heat of the material at x, and ρ(x) is its density. Fourier's law of conduction gives a relation between temperature u(x, t) and flux ϕ(x, t):

where σ 0 represents conductivity of the material. Substituting Equations ( 2) and (3) into Equation (1), we get,

where F is the heat received by the material per unit time per unit volume. Substituting k = σ0 cρ and β = 1 cρ we get,

If we inject heat energy f = f external per unit time per unit volume, and ϵ hs = ϵ HeatSource is the material absorptivity for the wavelength corresponding to heat source, then F = ϵ hs •f . This denotes that the heat F absorbed by the material is a fraction of the heat f that is supplied. Our goal is to estimate k, ε from the observations u(x, t) measured with a thermal camera. However, a thermal camera can only measure radiance and not the temperature directly. To solve this, we start by assuming that the absorptivity of the object is ε = 1 which gives us the radiation of an ideal black-body, u C = u black-body . The relationship between the real and ideal black body temperature is σu 4 black-body = σεu 4 , where σ is the Boltzmann constant. This gives us,

We now have an expression relating the measurements of the thermal camera u C , the input light source f , and the properties of the material at each spatial location ε and k.

Our goal is to use the measurements to estimate a spatial distribution of the thermal properties, ε(x, y) and k(x, y). Equation (6a) is an underconstrained problem as it has two unknowns for every equation. To regularize the problem, we assume that the diffusivity and absorptivity are constant over a small neighborhood giving us,

where ϵ ′ = β•ϵ hs ε 1/4 . Using the following shorthand notations:

Equation (6b) can be written as:

We derived all the equations above for the onedimensional case for simplicity, but the analysis can be easily extended to 3 dimensions. By replacing the Laplacian term u xx from 1-D Equation (7b) by the 3-D Laplacian ∆u xyz = u xx + u yy + u zz , we obtain:

Equation ( 8) is central to material classification proposed in this paper. The term k denotes thermal diffusivity of a material and ϵ ′ is a factor that depends on absorption (in heat source domain) and emission (in LWIR domain). The tuple of parameters forms a unique signature for materials that we use for the downstream task of classification.

We use an external source that is switched on for a known t ON duration and switched off. Let x = (x, y, z) be the combined spatial variable. We model f (x, t) as:

where f s (x) is the spatial profile of the external heat source.

For simplicity, we make the following assumptions:

• Our object shape is known and has a flat surface. This constraint can be relaxed by using a method like Structured Light to get the shape of object • The flat surface of the object faces the camera such that (x, y) coordinates lie in the plane of the surface and z lies perpendicular to it with z = 0 being at the surface and increasing as we go inside • For a given (x, y) the initial temperature is constant for all z, that is,

A spatially-uniform, known-intensity heat source allows us to estimate ϵ ′ from the temporal profile of heating and cooling. Estimation of k, however, requires spatial variation in temperature so that diffusion can occur. This can be seen from the Laplacian term, which will be zero across a patch of uniform temperature. This problem, in the absence of source, is the diffusion equation, and hence diffusivity recovery from a uniform-temperature patch is mathematically a total capture duration of 20s or higher, detailed analysis on this is presented in the supplementary. Our algorithm can also handle multiple simultaneous scans as shown in Fig. 8(a) We placed a graded ruler in the scene next to the target objects to calibrate the physical size of each pixel. We perform this calibration every time we move the setup, but on average our value of spatial ∆x was 0.5 mm (same for all dimensions). This process could be automated using structured light methods (refer Sec. 6). We code the differentiable FD algorithm in python using PyTorch [24]. We use an ADAM classifier with a decreasing learning rate for 400 epochs. Our system consisting of an RTX 3060, takes 5 minutes for the process to complete and recover the k and ϵ ′ maps. We captured measurements for each material in different initial temperatures and orientations. The code along with the dataset has been released.

In this section, we analyze the experimental observations made in lab and look at the performance of our parameter recovery and classification algorithms.

Similar to [10], we chose a few materials from the coarse categories of fabric, wood, paper, and metals. We made a dataset consisting of 22 materials or subclasses, applied our recovery algorithm on the data and used the generated features to train the classifier. Since metals have very low absorption and high diffusion, their TSF variation tends to have a very low magnitude. Since the ϵ ′ values are almost zero for metals, the FD algorithm did not converge for them. Although we cannot differentiate between different metals, this can be used as an indicator for metals. The data in Fig. 6(a) for metals validates the theory for metals. Barring the conducting materials, we performed FD analysis on the other materials and present our analysis on them.

Fig. 6 shows a plot of k vs ϵ ′ sampled at the center pixels of their recovered maps. For each measurement, we generate a feature vector by sampling k vs ϵ ′ in a window of specified size around the center. This feature vector along with the material label forms our training set for the classification algorithm. We perform a leave-one-out cross validation approach to obtain the classification accuracy of our method. The classification results can be found in the con-fusion matrix in Fig. 7(b). We obtain an overall accuracy of 85.9% using a multi-layer perceptron (MLP) classifier with a feature size of 50 pixels. With more features, the subclasses become more separable compared to what we see in Fig. 6(b). We also compare performance with varying feature sizes and classifiers suitable for dataset Tab. 1.

Materials that have similar diffusivity, and similar absorptivity for red wavelength (used in our experiments), produce similar TSFs. This in turn makes the recovered properties of these materials similar to each other. Such an issue occurs for brown tissue and wool as seen in Fig. 6(b) and Fig. 7(b) where the algorithm gets confused between the two. This can be mitigated by using more lasers like red, green and blue which gives us a set of absorption coefficients enabling more efficient segregation.

Our current modeling of thermodynamics requires certain simplifying assumptions for accurate results, such as planar geometry and constant temperature on the surface and bulk of the objects. These assumptions have enabled robust classification results in our preliminary experiments, but can be relaxed to extend to complex shaped objects with varying temperature profiles. Our approach can be combined with approaches for estimating scene geometry such as structured light [25±27] to estimate material properties of complex scenes and will be pursued as promising future direction.

Our experiments covered a limited set of materials to show that their TSFs can be used for material classification. Collecting a larger dataset that covers varying surface color, roughness, and geometry will enable a more robust material classification and will be pursued as future direction. We used Finite Differences in the forward model of our analysis. This part can also be potentially replaced by other approaches such as Physics Informed Neural Networks (PINNs) [28±30]. We can also potentially perform joint estimation of shape and the material properties using thermal structured Light. A known pattern of laser dots on the scene can be leveraged to estimate the thermal properties using the heat diffusion as well as the shape using the texture that is generated.

An advantage of thermal material classification over other methods like spectroscopy is that analyzing the heat diffusion can give us information that is hidden to the eye (even the thermal one). Consider a system as shown in Fig. 8(b), where the top layer is a different material from the one below it. When we inject heat, the upper layer ab-