ConcurrentHashMap is hash table supporting full concurrency of retrievals and adjustable expected concurrency for updates. I recently came across this code during testing, and one part really got my attention. To generate the hash, ConcurrentHashMap uses an algorithm based on bitshifting and bitwise operations.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

========================================
Variant of single-word Wang/Jenkins hash
========================================
private static int hash(int h) {
 
// Spread bits to regularize both segment and index locations,
// using variant of single-word Wang/Jenkins hash.
h += (h <<  15) ^ 0xffffcd7d;
h ^= (h >>> 10);
h += (h <<   3);
h ^= (h >>>  6);
h += (h <<   2) + (h << 14);
 
return h ^ (h >>> 16);
}

According to the comment in the code, this method applies a supplemental hash function to a given hashCode, which defends against poor quality hash functions.

Good hash functions are important as a hash table effectively turns from a map to a linked list, in the worst case, all keys in the same bucket. There are also other considerations that come into play such as the performance of hash calculation and the number of buckets. Dr. Heinz M. Kabutz explains the power of “power-of-two number of buckets” which gives us some good starting point to understand what is really going on here.

Let’s look at the code above and see how things change, line-by-line. To make things simple, I use int 1 to perform all the operations.

In Java, the int data type is a 32-bit signed two’s complement integer. To represent int 1 in binary code, we have the following:

1

h=1 > 0000-0000-0000-0000-0000-0000-0001

Now, let’s dissect the following line:

1

h += (h << 15) ^ 0xffffcd7d

First, let’s re-write this into an easier-to-read format.. at least for me .

1
2
3
4

h1 = h << 15      =  0000-0000-0000-0000-1000-0000-0000-0000
hex = 0xffffcd7d  =  1111-1111-1111-1111-1100-1101-0111-1101
h2 = h1 ^ hex     =  1111-1111-1111-1111-0100-1101-0111-1101
h2 + h            =  1111-1111-1111-1111-0100-1101-0111-1110

Using the same thought processing and applying it to each line, we end-up with:

1
2
3
4
5
6

h += (h << 15) ^ 0xffffcd7d = 1111-1111-1111-1111-0100-1101-0111-1110
h ^= (h >>> 10)	            = 1111-1111-1100-0000-1011-0010-1010-1101
h += (h << 3)		    = 1111-1101-1100-0110-0100-1000-0001-0101
h ^= (h >>> 6)              = 1111-1110-0011-0001-0101-0001-0011-0101
h += (h << 2) + (h << 14)   = 0100-1011-0100-0011-1101-0110-0000-1001
h ^= (h >>> 16)             = 0100-1011-0100-0011-1001-1101-0100-1010

Result:
Bin = 0100-1011-0100-0011-1001-1101-0100-1010
Decimal = 1,262,722,378

Conversion Table

Data Types and Data Structures

Creating a patch file

Creating a patch file is really easy. First, check out the most recent version of the code from Subversion using the ‘checkout’ command.

Make your changes. Then, in the root the project run the following command. It will store the patch file in your home directory. Make sure to give it meaningful filename.

svn diff > ~/fix_bug.diff

The file has the .diff extention, which stands for differences. This extension is recognized by many text editors and enables ’syntax highlighting’ automatically. (Give it a try with TextMate and you’ll know what I mean.) You can send the diff-file to the author of the project by email, or you can create a ticket in Trac and add it as an attachment. The author will review the changes you made and possibly apply them to the source.

Applying a patch

You should never apply patches from any person other than your development team without first reading through the changes, apply them locally and test your application and then commit them. Patches can not only include bug fixes, but also alterations to create back doors or add other exploits to your code.

Always read through a patch before applying it!

When you are sure the patch will bring no harm to you, your application or your customers, go ahead an apply it to your working copy. Here, I assume that you downloaded the patch file we previously generated, and placed it in your home directory. In the root of your application now run:

patch -p0 -i ~/fix_bug.diff

This will apply all the changes in the patch to your source. The -p0 option makes sure that all files can be found correctly (this has to do with something called ‘zero directories’, I won’t get into that right now). The -i option tells ‘patch’ what to use as input, in this case the ‘fix_bug.diff’ file in your home directory.
With the code changes in place, run your tests and make sure everything works as expected. If it does, commit your changes.

JProfiler helps you find performance bottlenecks, pin down memory leaks and resolve threading issues in your Java application. JProfiler combines CPU, Memory and Thread profiling in one application and is developed by ej-technologies. The latest version at the time of this article is 6.0.

The new 5.5 is now available with “Semisynchronous Replication”. This comes as an addition to the built-in asynchronous replication. MySQL replication is asynchronous by default. Events written to the binary logs on the Master server being retrieved by the slave server(s). Unfortunately, the Master server has no knowledge of when the slave has retrieved or processed these events. As a result, when Master crashes, transactions committed on Master, might have not have been committed on the slave server(s). In other words, there is no guarantee that any event will ever reach any slave.

In case of “Semisynchronous Replication”, the Master server blockes the transaction commit until at least one of the slave servers acknowledges the it has received all the events for that transaction. Obviously, this is great for consistency, however it brings up questions about replication over the WWW which isn’t an uncommon practice. In case of hitting the timeout limit (in case of no response from any of the slave servers), the Master server reverts back to asynchronous replication.

From MySQL support:

To understand what the “semi” in “semisynchronous replication” means, compare it with asynchronous and fully synchronous replication:

* With asynchronous replication, the master writes events to its binary log and slaves request them when they are ready. There is no guarantee that any event will ever reach any slave.
* With fully synchronous replication, when a master commits a transaction, all slaves also will have committed the transaction before the master returns to the session that performed the transaction. The drawback of this is that there might be a lot of delay to complete a transaction.
* Semisynchronous replication falls between asynchronous and fully synchronous replication. The master waits after commit only until at least one slave has received and logged the events. It does not wait for all slaves to acknowledge receipt, and it requires only receipt, not that the events have been fully executed and committed on the slave side.

Software doesn’t just appear on the shelves by magic. That program shrink-wrapped inside the box along with the indecipherable manual and 12-paragraph disclaimer notice actually came to you by way of an elaborate path, through the most rigid quality control on the planet. Here, shared for the first time with the general public, are the inside details of the program development cycle.

1. Programmer produces code he believes is bug-free.
2. Product is tested. 20 bugs are found.
3. Programmer fixes 10 of the bugs and explains to the testing department that the other 10 aren’t really bugs.
4. Testing department finds that five of the fixes didn’t work and discovers 15 new bugs.
5. See 3.
6. See 4.
7. See 5.
8. See 6.
9. See 7.
10. See 8.
11. Due to marketing pressure and an extremely pre-mature product announcement based on overly-optimistic programming schedule, the product is released.
12. Users find 137 new bugs.
13. Original programmer, having cashed his royalty check, is nowhere to be found.
14. Newly-assembled programming team fixes almost all of the 137 bugs, but introduce 456 new ones.
15. Original programmer sends underpaid testing department a postcard from Fiji. Entire testing department quits.
16. Company is bought in a hostile takeover by competitor using profits from their latest release, which had 783 bugs.
17. New CEO is brought in by board of directors. He hires programmer to redo program from scratch.
18. Programmer produces code he believes is bug-free….

Verbose GC options

Parallel Scavenge Collector

Parallel Copying Collector

Profiling Queries with SHOW STATUS
mysql> flush status;
mysql> select SQL_NO_CACHE count(*) from table;

- Check query plan now:

mysql> show status like 'Select%';

- Check engine operations:

mysql> show status like 'Handler%';

- Check if there was any ordering:

mysql> show status like 'Sort%';

- Check how many temporary tables have been created:

mysql> show status like 'Created%';


Profiling Queries with SHOW PROFILE

mysql> set profile=1;
select count(*) from table;
mysql> show profiles\G
*************************** 1. row ***************************
Query_ID: 1
Duration: 9.28089300
Query: select count(*) from table
1 row in set (0.00 sec)
mysql> show profile;
+--------------------+----------+
| Status | Duration |
+--------------------+----------+
| starting | 0.165418 |
| Opening tables | 0.000024 |
| System lock | 0.000004 |
| Table lock | 0.000008 |
| init | 0.000014 |
| optimizing | 0.000006 |
| statistics | 0.000013 |
| preparing | 0.000012 |
| executing | 0.000006 |
| Sending data | 9.115348 |
| end | 0.000015 |
| end | 0.000004 |
| query end | 0.000003 |
| freeing items | 0.000008 |
| closing tables | 0.000004 |
| logging slow query | 0.000002 |
| cleaning up | 0.000004 |
+--------------------+----------+

mysql> show profile cpu;
+——————–+———-+———-+————+
| Status | Duration | CPU_user | CPU_system |
+——————–+———-+———-+————+
| starting | 0.000063 | 0.000000 | 0.000000 |
| query end | 0.000004 | 0.000000 | 0.000000 |
| freeing items | 0.000005 | 0.000000 | 0.000000 |
| logging slow query | 0.000003 | 0.000000 | 0.000000 |
| cleaning up | 0.000003 | 0.000000 | 0.000000 |
+——————–+———-+———-+————+

Is a certain process running your CPU right into the ground? How do you find said process without picking your way through the ps aux results? With this command:

ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu | sed '/^ 0.0 /d'

…at which point you can kill it with sudo kill -9.