Truth be told, ModSecurity does not have a random function. If we need random numbers, we need to get them
ourselves. Josh Zlatin has shown how you can do this with lua. If we do not want to use lua, we are left on your own devices.
I am not in need of cryptographically secure random numbers. What I need is a way to do conditional
branching based on random numbers. Say you start to deploy the Core Rules on a heavy traffic site. Enabling
them will likely drown you in false positives. How about enabling them for 1% of the requests and tune
the basics first? Then you slowly raise the percentage of requests you funnel into the core rules…
Or you have a rule, which you do not trust entirely, but you need production traffic to test it. It would
be a pity to break the site, but maybe you can start with enabling the rule for every 10th request?
So my need for randomness is mostly sampling used to do conditional rule execution.
But how can we do this? What sources of entropy can we use? And how do we transform the entropy into a
value which can be used in a SecRule statement?
I see two sources of entropy. One is the unique-id of the
request, which has some randomness included. The second one is the age of the request in microseconds. If you
take but the last digit of that number, that is quite random. Take the second digit from the right and you
start to see a clustering (at least I did in my tests).
The problem with the unique-id is, that the entropy is distributed across the whole id, mixed with static
parts. Most of the data is not numeric, which complicates things further. But we can transform the id
with sha1 or md5 to distribute the entropy evenly. If we do this, we can – for example – take the first
digit and use that as a random number.
With the duration we can use the same technique. The last digit of the duration is okay, but the rest of
the entropy is best extracted via a hashing function and then the first digit.
Unfortunately, a new problem arises with this hashing: there is no guarantee that sha and md5
hashes contain digits. So we could fall in the gap with some of our requests.
As kind of a fallback, I use the final digit of the epoch, that is the seconds since the start of the
unix time. This combined with the final digit of the duration gives me a guarantee, that I have at least
two digits of randomness.
Here is the list of digits, I am using:
* First digit in the SHA1-hashed unique id (data not guaranteed)
* First digit in the SHA1-hashed duration (microseconds; data not guaranteed)
* First digit in the md5-hashed unique id (data not guaranteed)
* First digit in the md5-hashed duration (microseconds; data not guaranteed)
* Last digit in the duration (microseconds; data guaranteed)
* Last digit in the epoch (seconds; data guaranteed)
Ideally, I am getting a number with six digits. In the worst case, I am getting only two digits
of a bit weaker quality. My approach is then to take the first digit of this number and fill
it into an environment variable RND10 and to take the first two digits and fill them into RND100.
Here is the recipe, which also removes the leading “0” from RND100.
# === Calculating Random Numbers
#
# ATTENTION: These are pseudo-random numbers. There is no security value in these numbers.
#
# Env variable RND10: Range from 0-9
# Env variable RND100: Range from 0-99
#
# Entropy taken from unique id, duration of request in microseconds and unix epoch.
#
SecRule UNIQUE_ID "([0-9])" "id:80000,phase:1,pass,nolog,capture,t:none,t:sha1,setvar:TX.r1=%{TX.1}"
SecRule DURATION "([0-9])" "id:80001,phase:1,pass,nolog,capture,t:none,t:sha1,setvar:TX.r2=%{TX.1}"
SecRule UNIQUE_ID "([0-9])" "id:80002,phase:1,pass,nolog,capture,t:none,t:md5,setvar:TX.r3=%{TX.1}"
SecRule DURATION "([0-9])" "id:80003,phase:1,pass,nolog,capture,t:none,t:md5,setvar:TX.r4=%{TX.1}"
SecRule DURATION "([0-9])$" "id:80004,phase:1,pass,nolog,capture,t:none,setvar:TX.r5=%{TX.1}"
SecRule TIME_EPOCH "([0-9])$" "id:80005,phase:1,pass,nolog,capture,t:none,\
setvar:TX.rndtmp=%{TX.r1}%{TX.r2}%{TX.r3}%{TX.r4}%{TX.r5}%{TX.1}"
SecRule TX:rndtmp "^([0-9])" "id:80006,phase:1,pass,log,capture,t:none,\
msg:'Random number RND10: %{TX.1}',setenv:RND10=%{TX.1}"
SecRule TX:rndtmp "^0" "id:80007,phase:1,pass,nolog,skip:1"
SecRule TX:rndtmp "^([0-9][0-9])" "id:80008,phase:1,pass,log,capture,t:none,\
msg:'Random number RND100: %{TX.1}',setenv:RND100=%{TX.1},skip:1"
SecRule TX:rndtmp "^.([0-9])" "id:80009,phase:1,pass,log,capture,t:none,\
msg:'Random number RND100: %{TX.1}',setenv:RND100=%{TX.1}"
I have tested these numbers and got fairly decent stats. Let’s say good enough for my use:
Distribution of RND10 (based on a sample of 100.000 requests):
0 9.95% 1 9.78% 2 10.06% 3 10.00% 4 10.14% 5 10.09% 6 9.94% 7 10.17% 8 9.93% 9 9.95%
Distribution of RND100 (based on a sample of 100.000 requests):
0 0.97% 1 0.97% 2 0.99% 3 0.98% 4 0.98% 5 0.99% 6 1.01% 7 1.04% 8 1.02% 9 1.00% 10 0.99% 11 0.97% 12 0.97% 13 0.98% 14 0.94% 15 0.95% 16 0.98% 17 0.98% 18 1.00% 19 1.00% 20 1.02% 21 0.99% 22 0.98% 23 0.94% 24 1.07% 25 0.99% 26 1.02% 27 1.01% 28 1.03% 29 1.02% 30 1.02% 31 0.96% 32 1.00% 33 1.02% 34 1.03% 35 1.01% 36 0.97% 37 0.98% 38 1.01% 39 1.00% 40 0.97% 41 0.97% 42 1.05% 43 1.01% 44 1.02% 45 1.04% 46 1.00% 47 1.03% 48 0.99% 49 1.05% 50 1.02% 51 0.97% 52 0.97% 53 1.04% 54 1.03% 55 0.99% 56 1.03% 57 0.99% 58 1.00% 59 1.03% 60 0.98% 61 1.02% 62 0.99% 63 0.96% 64 1.00% 65 1.03% 66 0.99% 67 0.97% 68 1.00% 69 1.00% 70 0.96% 71 1.01% 72 1.07% 73 1.01% 74 1.02% 75 1.05% 76 1.02% 77 1.03% 78 0.99% 79 1.01% 80 1.01% 81 0.96% 82 1.00% 83 0.98% 84 1.00% 85 0.98% 86 0.97% 87 1.02% 88 1.00% 89 1.00% 90 1.04% 91 0.99% 92 0.99% 93 0.99% 94 0.98% 95 1.01% 96 0.93% 97 1.01% 98 1.01% 99 1.01%
Here is how I use this in practice. I can adjust the percentage of requests filtered by the
core rules via a variable named tx.percentage_reqs_crs. In this example, the core rules are
executed for 5% of the requests.
# === Calculating Random Numbers
#
# ATTENTION: These are pseudo-random numbers. There is no security value in these numbers.
#
# Env variable RND10: Range from 0-9
# Env variable RND100: Range from 0-99
#
# Entropy taken from unique id, duration of request in microseconds and unix epoch.
#
SecRule UNIQUE_ID "([0-9])" "id:80000,phase:1,pass,nolog,capture,t:none,t:sha1,setvar:TX.r1=%{TX.1}"
SecRule DURATION "([0-9])" "id:80001,phase:1,pass,nolog,capture,t:none,t:sha1,setvar:TX.r2=%{TX.1}"
SecRule UNIQUE_ID "([0-9])" "id:80002,phase:1,pass,nolog,capture,t:none,t:md5,setvar:TX.r3=%{TX.1}"
SecRule DURATION "([0-9])" "id:80003,phase:1,pass,nolog,capture,t:none,t:md5,setvar:TX.r4=%{TX.1}"
SecRule DURATION "([0-9])$" "id:80004,phase:1,pass,nolog,capture,t:none,setvar:TX.r5=%{TX.1}"
SecRule TIME_EPOCH "([0-9])$" "id:80005,phase:1,pass,nolog,capture,t:none,\
setvar:TX.rndtmp=%{TX.r1}%{TX.r2}%{TX.r3}%{TX.r4}%{TX.r5}%{TX.1}"
SecRule TX:rndtmp "^([0-9])" "id:80006,phase:1,pass,log,capture,t:none,\
msg:'Random number RND10: %{TX.1}',setenv:RND10=%{TX.1}"
SecRule TX:rndtmp "^0" "id:80007,phase:1,pass,nolog,skip:1"
SecRule TX:rndtmp "^([0-9][0-9])" "id:80008,phase:1,pass,log,capture,t:none,\
msg:'Random number RND100: %{TX.1}',setenv:RND100=%{TX.1},skip:1"
SecRule TX:rndtmp "^.([0-9])" "id:80009,phase:1,pass,log,capture,t:none,\
msg:'Random number RND100: %{TX.1}',setenv:RND100=%{TX.1}"
# === ModSecurity Core Rules Inclusion
SecAction "id:'10000',phase:1,t:none,setvar:tx.percentage_reqs_crs=5"
SecRule ENV:RND100 "!@lt %{tx.percentage_reqs_crs}" "id:10001,phase:1,pass,nolog,t:none,skipAfter:END_CRS_INCLUDE"
SecRule ENV:RND100 "!@lt %{tx.percentage_reqs_crs}" "id:10002,phase:2,pass,nolog,t:none,skipAfter:END_CRS_INCLUDE"
SecRule ENV:RND100 "!@lt %{tx.percentage_reqs_crs}" "id:10003,phase:3,pass,nolog,t:none,skipAfter:END_CRS_INCLUDE"
SecRule ENV:RND100 "!@lt %{tx.percentage_reqs_crs}" "id:10004,phase:4,pass,nolog,t:none,skipAfter:END_CRS_INCLUDE"
SecRule ENV:RND100 "!@lt %{tx.percentage_reqs_crs}" "id:10005,phase:5,pass,nolog,t:none,skipAfter:END_CRS_INCLUDE"
Include /core-rules/*.conf
SecMarker END_CRS_INCLUDE
This works nicely and should help people who want to ease into the core rules or a similar set of rules.
If you think there is a misconception somewhere or if you see a better source of entropy, then please let me know.
Of course, you can also spoil the whole fun by providing a patch for a ModSecurity random function.
Christian Folini Follow @ChrFolini Tweet