Line segment detection is essential for high-level tasks in computer visionand robotics. Currently, most stateof-the-art (SOTA) methods are dedicated todetecting straight line segments in undistorted pinhole images, thusdistortions on fisheye or spherical images may largely degenerate theirperformance. Targeting at the unified line segment detection (ULSD) for bothdistorted and undistorted images, we propose to represent line segments withthe Bezier curve model. Then the line segment detection is tackled by theBezier curve regression with an end-to-end network, which is model-free andwithout any undistortion preprocessing. Experimental results on the pinhole,fisheye, and spherical image datasets validate the superiority of the proposedULSD to the SOTA methods both in accuracy and efficiency (40.6fps for pinholeimages). The source code is available athttps://github.com/lh9171338/Unified-LineSegment-Detection.