Simple bacterial gene regulatory motifs can be viewed from the perspective of dynamical systems theory, and mathematical models of these have existed almost since the statement of the operon concept. In this talk I will review the three basic types of these regulatory mechanisms and the underlying dynamical systems concepts that apply in each case. In the latter part of the talk, I will discuss the exciting mathematical challenges that arise when trying to make honest mathematical models of the underlying biology. These include transcriptional and translational delays, and the fact that these delays maybe state dependent, as well as the interesting and often unsolved problems of characterizing the noise inherent in bacterial dynamics. Surprisingly, the inclusion of state dependent delays can have rather profound effects on the possible dynamical behaviour, and it is unclear if this is reflected in biological behaviour.