The Collision Avoidance (CA) system constitutes a key enabling technology for the Maritime Autonomous Surface Ships (MASS), the appropriate functionality of which is critical for assuring the navigation safety. Although several techniques including testing of the collision scenarios in a virtual environment can be employed, the trust of testing phase results depends on the number of tested scenarios and their coverage. This study aims at proposing a systematic and automatic process for the generation of the traffic scenarios that can be employed for the CA system testing. First, the range of the investigated parameters is defined, and samples of potential traffic parameters are generated using Sobol sequences. Subsequently, hazardous traffic scenarios are identified from the initially generated scenarios by using predefined rules. For these hazardous scenarios, a risk vector considering weather conditions and traffic conditions is calculated. A clustering algorithm is employed to identify the groups of traffic conditions that can be encountered based on each scenario risk vector and COLREGs traffic scenarios. For each of these groups, the riskiest scenario is provided as input for the test cases development, thus, simplifying the selection process of testing scenarios. The process is applied to a theoretical Short Sea Shipping autonomous vessel, whereas the derived results are employed to discuss the advantages and disadvantages of the developed process.