The castle is first mentioned in written sources during the 1240s as a fortification of Salzburg Archdiocese, although a castle is believed to have existed at the site for many centuries before this. It fell to the Habsburgs in the 1490s but was subsequently burned to the ground twice, in the peasant uprisings of 1515 and 1528. It was rebuilt between 1531 and 1550 to an Italian design, featuring four defence towers and two defence walls. In subsequent centuries the castle was owned by the Gallensteins, the Frankopans and finally the Attem Counts, who sponsored major decorative work by Baroque artists from Styria during the period 1694-1732. The Attems continued to own the castle until 1945, although by the early 20th century they were no longer resident. The castle (including its decorative stucco work) suffered severe damage in the earthquake of 1917, and further damage was caused in the course of subsequent work to accommodate office, apartments and (during World War I) a military hospital for the wounded from the Soča Front. After the World War II the castle became the headquarters of Vino Brežice and home to 26 families. The Museum of the Posavje Region, Brežice moved to the castle in 1949.
Brežice Castle is an excellent example of fortified Renaissance castle architecture on a plain. Its ground plan is a slightly irregular rectangle, having its corners fortified with powerful round defence towers, in some places as thick as 4 metres, equipped with numerous crenels. The building is surrounded by a spacious bailey, landscaped as a park with a fountain in the middle.
The castle building itself has two storeys. The heavily-fortified ground floor is made of cut stone in the shape of an inclined wall, becoming vertical at the level of the first floor; the inclined and the vertical parts are divided by a stone protuberance. The west wall of the castle, between the north and south defence towers, has at the bailey side on both stories long arcade corridors with elegantly-shaped Tuscan columns. The south east and the west tower are connected on the first and second floor by a narrow corridor with preserved crenels. At the bailey side of the north and west castle wing there are two sundials painted on a large wall surface. The first measured time in the morning, the second in the afternoon.
The interior of the castle has been a luxurious Baroque residence since the times of the early Attem Counts. The focus of the east wing is the Great Hall, named also the Knight's Hall. The largest Baroque painted secular room in Slovenia measures 35 metres long by 10 metres wide and 8 metres high and is decorated with images from Ovid's Metamorphoses and other legends (1699–1703) by Frančišek Karl Remb. In the west wing a Renaissance staircase was built featuring frescoes of Hercules' story (1721–1727) by Franz Ignac Flurer, who also decorated the castle chapel with frescos of the Glorification of the Cross. The altar in the chapel is from the workshop of Luka Mislej and dates from around 1718.
Brežice Castle served as a picturesque backdrop for some international film productions in the 80s, also due to the former Yugoslavia's enterprising film studios, e.g. Jadran Film located in Zagreb. Undoubtedly, the most famous example presents a 1990 comedy-drama film Rosencrantz & Guildenstern Are Dead, written and directed by Tom Stoppard. A visual display of castle interiors, especially the Knight's Hall or wine cellar, add colour to punny scenes with memorable quotes of Tim Roth as Guildenstern and Gary Oldman as Rosencrantz.