In order to understand how mining difficulty works, you must know what mining is.
Bitcoin’s networks have miners who use specialized computers to solve cryptographic hard puzzles and if they are successful they get to add blocks in the blockchain network. Basically there are transactions waiting in the mempool for miners to hash. Then the miners add a random value to the front of the hash and hashes the whole value. That hash should be less than a particular value called difficulty.
If you remember right, bitcoin network has a set block time of 10 min, meaning that a new BTC can be added into the supply in every 10 minutes. The increase or decrease in the level of difficulty of bitcoin mining with respect to the mining ease. So when mining is easy for miners, the network difficulty increases and vice versa. With a set universal block difficulty, blocks should have a hash rate that is below the goal.
Hash rate is used to judge the proof of work network, determining the strength of the miners inside the network. Hash rate also has a direct relation with the security and speed of the network. Higher the network’s hash rate, higher the overall security.
In order to keep a consistent production of blocks, hash rate should be in control. That is why BTC difficulty increases when the overall hash rate of the network becomes high which makes it harder for miners to mine. The same phenomena is also valid in the opposite situation. Decrease in the bitcoins hash rate will cause reduction in the network difficulty.
Miners use special machines that are extremely fast in producing tetrahashes per second. These are ASIC hardware specialized to mine bitcoins. With the grand number of miners participating in the network, it would be extremely hard and non sensible for a system to check each one of them to see if they meet the necessary requirements. To check all the hashes produced by every miner per second would be completely impractical, especially in a bitcoin mining pool.
Difficulty is changed with every 2016 block depending on the time it took to find the previous ones. By fact, if it takes 10 min to discover one block then it will take 2 weeks in finding 2016 blocks. So lets say if it took more than 2 weeks for miners to discover the previous 2016 blocks, the mining difficulty would be decreased as in inverse, if it takes less than two weeks the difficulty would be increased. The faster the 2016 blocks are mined the higher the difficulty is raised.
With every block mined, miners receive BTC reward for processing transactions which is halved in almost every 4 years after the successful mining of 210,000 . The profits of the miners directly relates to their contribution of hashing power to the system. So if the mining power remains constant and the network’s hash power increases, the miner’s share of hashing power automatically decreases in the network. Difficulty as discussed above is based on the rate at which the previous blocks were mined to maintain the consistency in the network. However the increase in difficulty is correlated with the value of BTC which helps in better compensation for miners because if miners abandon the network, it loses transaction completion and security of the network. So even with many raised eyebrows over mining rewards with every halving, it seems unlikely for miners to go unpaid.
With a decentralized ecosystem of bitcoin of wide network reporting to no central authority or entity, protocols ensure fair regulation to maintain some form of governance. This is where the Nakamoto consensus comes into play. It clearly defines that in order to participate in the network one needs to pay. In this case it is in the form of proof of work, referencing the amount of computational power a miner must spend to mine one bitcoin. This is where difficulty comes in the picture that makes the mining hard solving the double spending problem and keeps the supply of BTC consistent.