Natural disasters have been always caused a danger to human life, and among these are earthquakes. Seismic risk assessment consists of the evaluation of existing buildings and their expected response in case of an earthquake; the exposure model of buildings has a significant role in the final results of risk calculations. With this respect, several studies, including traditional data acquisition (e.g. visual survey) or advanced methods (e.g. remote sensing and machine learning) are conducted. In recent years, advanced techniques have been developed to speed up and automatize the processes of data acquisition to data interpretation, although it is worth mentioning that the visual survey is essential to train and validate machine learning methods. In the present study, the traditional visual survey is combined with the implementation of a deep learning model to identify building types. As a first outcome, city mapping schemes are obtained by classifying buildings according to the main features (i.e., construction period and height classes). Next, Random Forest (RF), a supervised learning algorithm, is applied to classify buildings into building types by exploiting all the building attributes. The RF model, trained and tested on the cities of Neuchatel and Yverdon-Les-Bains and then applied to two other Swiss cities, which are also visually/physically (e.g. Google street) surveyed. The decent accuracy of the results by application of the model to two cities with different distributions of building types showed the robustness of the method in the building types classification in other cities in Switzerland, paving the path for its application to the whole country. Finally, to study the performance of the proposed building type detection in seismic risk assessment, the seismic damage for two different scenarios is evaluated by considering the real and predicted building exposure models. A negligible discrepancy between the estimated damages based on the real and predicted exposure models demonstrate the successfulness of the method in risk assessment with high accuracy.