A key way in which the Nazi government controlled the people of Germany was through fear.

The Gestapo (the secret police run by Heydrich) had a network of informers who reported any disloyalty or opposition to the regime.
The SS was run by Himmler. They set up concentration camps which were large prisons. People who opposed were either sent to concentration camps or could be shot. About 500 000 Germans were killed.
The police and the judiciary were controlled by the Nazis. They targeted opponents of the regime but overlooked crimes by Nazis.
During WWII the SS formed the Einsatzgruppen (squads of soldiers that shot 1 million Jews) and later the Death Head Units that operated the Extermination camps.
The Waffen SS was an elite unit in the German army.

The system ran on fear. There weren't that many Gestapo agents - about 20 for every million inhabitants. However informers were everywhere and no-one knew who they were - they could be neighbours, co-workers, shopkeepers or even members of your own family.

Terror was a key reason why very few people dared to oppose the Nazi regime because the risk was so great. For example, 13 Edelweiss Pirates were publicly executed in Cologne in November 1944. What opposition there was, had to be in small, secret groups.

GCSE Germany 1890 - 1945 AQA

Nazi Terror