Sources first mention the settlement in 1228 under the name of Schirnomel, it became a town in 1497, in the period of the Turkish invasions. The economic development of Črnomelj accelerated after the construction of the Novo mesto–Črnomelj–Karlovac railway, completed in 1914. In the interwar period, the town obtained its first steam-powered saw-mill and foundry, and a brown coal mine opened in Kanižarica. During World War II Črnomelj was occupied by the Italians. On 19–20 February 1944, the first assembly of SNOS, the Slovene Parliament, was held in Črnomelj's cultural hall. After the war, industry started to develop and bridges and buildings which had been destroyed were soon renovated or rebuilt.
The castle, situated in the historical centre of Črnomelj, has passed several phases of construction from its establishment in the 13th century. It incorporates a few Gothic portals. Today it is a two-storey four-winged building with an atrium and an accent at the west wing. There are three plaques built in the outer structure, commemorating the District Committee of the National Liberation, the Founding Assembly of the Slovene Association of Journalists, and Matija Malešič (b. 1933), a native lawyer and ambassador.
Besides Črnomelj Municipal Museum, the castle also houses the seat of the Črnomelj Municipality and Tourist Information Centre.
On the ground floor are presented Roman tombstones found on Ločki Okljuk. In two small rooms on the first floor, the history of the town of Črnomelj is shown from prehistoric times to the middle of the 20th century, with an emphasis on the town's social activities, economic development and industrialisation. Several items linked with the history of the town are displayed, including an installation of a worker's kitchen from the middle of the 20th century.
A permanent exhibition of works by Božidar Jakac (1899–1989) featuring Bela krajina motifs was opened in the gallery of the Črnomelj Municipal Museum in 1999.