Cross CPU

I use Bash scripts quite often for many internal tools. One of the scripts I use everyday now grows to about 300 lines. Its purpose is to auto fetch various internal data (such as multiple JIRAs) and organize the results into one single page, which saves me a lot of time. I have read the Google Shell Style long ago,  and finally got some time during Xmas break to follow the guidelines. Here are the major findings: Use  $(command)  instead of backticks. Backticks refers to `, for example results=`cmd arg1 arg2` The reasons not to use backticks anymore are due to backticks an be messy and not good for nesting. If you have backticks inside backtick, you would need to escape them. A good example here libdir=$(dirname $(dirname $( which gcc)))/lib libdir=`dirname \`dirname \\\` which gcc\\\`\``/lib $ is more safe and predictable.  Use (( ... )) or $(( ... )) instead of expr (command) . The reasons not to use expr is, it is simply too old. For the difference of (( ... )...

" The California License Plate Game ", or " Count to 24 ", is something I play while I am driving, particularly when stuck in traffic.  As you could see, the general California license plate, showing above, is made of " nXXXnnn" , where n is a number between 0 to 9 and X letter from A to Z. The game is to calculate to 24 with those 4 numbers by only using + - x /. For example. the 4 numbers above  [6 2 4 4], could arrive 24 as 2 *(6+4) + 4. Of course, there are certainly license plates that is impossible to do so, such as [6 2 5 0]. For a long time, I have been wondering what is the overall probability for a license plate to be able to count to 24 and also want to have answers for each and every possible ones. Finally this is the result of a script I ran:  https://github.com/a10jliu/licenseplategame/blob/main/Results .  Here are the results:     0.6902 The probability of a license place could count to 24.  That is 6902 among the total 10000 possible c...

For a long time, I had an Expect script to login network devices, fetch information and put the output together as a Web page. Think of it as a script checking router version, memory etc. It took about 30 seconds to finish checking a single target and would take hours to finish scanning several subnets. Finally I think it is time to parallel these tasks so the whole scan could take the same time as scan only one device (etc. 30 seconds versus hours).  And is excited to find out the simple and powerful usage of xargs -P option. Here are man xargs result regarding -P:     -P max-procs, --max-procs=max-procs Run  up  to  max-procs processes at a time; the default is 1.  If max-procs is 0, xargs will run as many processes as possible at a time.  Use the -n option or the -L option with -P; otherwise chances are that only one exec will be done. Put together into my small script: #!/usr/bin/bash echo >/tmp/list subnets="192.168.4...

I hate traffic so almost every night I would check traffic to home on Google Map before leaving work. It has saved me several times from stuck in traffic jam but  occasionally I forgot to do so and then found myself driving 20 mph in the highway. So one day I thought about this and decided to write a small AJAX script to update the traffic status in the front page of my Web browser every minute. Then by saving the data (from Google Map) and incorporate into Google Chart (thanks!), I got a daily traffic chart going on to monitor traffic both ways between home and work.   Now I have about 4 weeks' of data and here are some observations: Base time : ~ 20 mins between Home and Work when there is no/light traffic Rush hours :  Morning rush hour ( when traffic >30 mins ) is always around 7:20 to 10AM in weekdays Afternoon rush hours , however, starts about 20 mins earlier everyday from Monday to Friday, so it appears people have more work on Monday Mo...

BPF (Berkley Packet Filter) is a very powerful packet matching tool to quickly identify certain payload patterns. For example, a BPF could be created to ONLY allow TLS 1.2 request passing through while blocking all other lower version traffics. It takes several tries to finally figure out a filter working for most scenarios and here it is   tcp[tcp[12]/16*4+5]==0x01 and tcp[tcp[12]/16*4+9:2]!=0x0303 Explanations: 1 Since TCP header size could vary between 20 and 60 bytes (although most widely seen are 20 and 32), the exact offset to match the TLS version is different and need to locate based on the TCP header size. 2 tcp[12] is the TCP header size. 3 tcp[12]/16*4 will convert the TCP header length from HEX to decimal. For example. 0x50 results in 20 bytes while 0x80 results as 32 bytes. 4 offset + 5 will match the Client Hello (1) location and we need this to avoid accidentally match other handshake protocol types, such as client key exchange etc. 5 offset + 9...

  When deploying DDoS Mitigation, it is normal to enable certain kinds of client authentication method to determine a client is valid or not. For example, DDoS mitigator might use TCP authentication to prevent a SYN Flood attack: The first TCP connection from a client will never reach the intended server. The DDoS mitiigator will check the 3 way handshake behaving as the target server by responding a SYN ACK. Assuming only valid client will then proceed to response a ACK thus to finish the handshake, only then this client is considered a valid client. While mitigator will still RST the connection and forward all subsequent connections from the same client. If a client could not finish 3 way handshake (no ACK is received), mitigator could either try keep authenticating or put this client IP in a blacklist to drop all connections. But, when deploying multiple DDoS mitigators using ECMP, extra considerations are needed for any type of client authentications. ...

  It is a very desirable  function if you could specify source IP ( source Port will be extra useful) for network testing and  debugging. Here is a short list of command syntax for commonly used tools in Linux curl curl http://10.101.26.104 --interface   6.7.8.9 --local-port 23456 dig dig jian.p1.tps.a10lab.com @7.7.7.1 -b 50.35.26.100#23456 wget wget --bind-address=6.7.8.9   http://10.101.26.104 wget could not specify source port; but you could use the following trick to limit the source ports used in Linux: check the config of file /proc/sys/net/ipv4/ip_local_port_range cat /proc/sys/net/ipv4/ip_local_port_range 32768   60999 By default the source ports ranges from 32678 to 60999 change the file to limit to one source port echo "23456 23456" > /proc/sys/net/ipv4/ip_local_port_range Then test again, your wget command will use source port 23456 nslookup Does not have any option...

On April 1 2018, CloudFlare announced the Internet's fastest, privacy-first resolver 1.1.1.1. It also provide a directly Webpage at https://1.1.1.1. The interesting part is my browser actually trusts this site (as shown on the Firefox above) by visiting the target IP 1.1.1.1.  This is rare as almost all https are now using domain name based certificate to match the FQDN we input in browser address bar. We almost take for granted that input an IP address for https will result a certificate warning and I definitely read before that request for a certificate for IP is not allowed now, for example, here . So what is really happening here ? A quickly check the certificate shows the cert is valid for *.cloudflare-dns.com and although 1.1.1.1 does resolve to  1dot1dot1dot1.cloudflare-dns.com, this alone will not make browser to trust the site. Finally check the Certificate Subject Alt Name (SAN) extension s, we found out there are 4 IPv4/IPv6 addresses including th...