Ship collision avoidance has always been a concern and it is crucial for achieving safe navigation of ships at sea. There are many studies on ship collision avoidance in open water, but less attention on coastal waters considering the uncertainty of target ships due to the complexity of the environment and traffic flow. In this paper, collision avoidance decision-making research in coastal waters considering the uncertainty of target ships was proposed. Firstly, accurate ship trajectories are obtained by preprocessing the raw Automatic Identification System (AIS) data. Subsequently, the processed trajectories are clustered using the Ordering Points to Identify the Clustering Structure (OPTICS) algorithm and Hausdorff distance, acquiring a dataset for trajectory prediction of target ships. Then, a mixed Gaussian model is utilized to calculate the prior probability distribution of the prediction model, thus establishing a trajectory prediction model that considers the uncertainty of the target ship. Finally, ship maneuverability is simulated using the Mathematical Model Group (MMG) and Proportion Integration Differentiation (PID) models, and a collision avoidance decision-making model for ships is constructed. The proposed algorithm has been tested and verified in a case study. The results show that the approach effectively predicts the trajectory of the target ship and facilitates informed collision avoidance decision-making.