Conducting hazardous materials (HAZMAT) commodity flow studies (CFS) is crucial for emergency management agencies. Identifying the types and amounts of hazardous materials being transported through a specified geographic area will ensure timely response if a HAZMAT incident takes place. CFS are usually conducted using manual data collection methods, which may expose the personnel to some risks by them being subjected to road traffic and different weather conditions for several hours. On other hand, the quality and accuracy of the collected HAZMAT data are affected by the skill and alertness of the data collectors. This study introduces a framework to collect HAZMAT transportation data exploiting advanced image processing and machine learning techniques on video feed. A promising convolutional neural network (CNN), named AlexNet was used to develop and test the automatic HAZMAT placard recognition framework. A solar-powered mobile video recording system was developed using high-resolution infra-red (IR) cameras, connected to a network video recorder (NVR) mounted on a mobile trailer. The developed system was used as the continuous data collection system. Manual data collection was also conducted at the same locations to calibrate and validate the newly developed system. The results showed that the proposed framework could achieve an accuracy of 95% in identifying HAZMAT placard information. The developed system showed significant benefits in reducing the cost of conducting HAZMAT CFS, as well as eliminating the associated risks that data collection personnel could face.