The BLAST benchmarks were developed by George Coulouris and others from NIH. They are aimed at workstations and small clusters. The BLAST benchmark simulates typical workload obtained from analysis of several hundred thousand runs at NIH.
Applications
- Evaluate the relative performance of BLAST running on different hardware
- Evaluate the relative performance of different BLAST implementations
Tasks
Program distribution:
blastn (Nucleotide-nucleotide BLAST) 50%
megablast (Large numbers of query sequences) 10%
blastp (Protein-protein BLAST) 20%
blastx ( nucleotide query sequence against a protein sequence database) 10%
tblastn ( protein query against the six-frame translations of a nucleotide sequence database) 5%
tblastx (nucleotide query against the six-frame translations of a nucleotide sequence database) 5%
Results {Version 2.2.13} (legacy times, deprecated)
Dual Xeon 3.6 GHz - 885 sec. or 14:45 min
Dual Opteron 2.8 GHz - 930 sec. or 15:30 min (Win32, Dual Opteron 254 2.8 Ghz - ARECA 1120 RAID5)
Results {Version BLAST+ Benchmark Suite 2013}
Elapsed Time (short) : 02:15 min (Win64, Dual Xeon E5-2687W 3.4 GHz (16 core, 32 threads , 50 GByte RAMDISK)
Elapsed Time (long) : 15:01 min (Win64, Dual Xeon E5-2687W 3.4 GHz (16 core, 32 threads , 50 GByte RAMDISK)
Details for BLAST+ Benchmark Suite 2013
The current short implementation (August 2013 with parallel make) is not efficient for the performance evaluations of modern workstations, because the sequential order of the commands requires long wait times for certain threads. That is especially true for computations on computers with higher number of available threads (n>=32). One single tblastx process (XM_194279) is responsible for running 99% of the time, with over 200 other queries already finished. Removing this thread the total computation time shortens by 25% to 1:32 min. So the benchmark is rather a test for high single thread speed. A corrected Windows make file that resolves some of the missing files and path issues (but does not solve the XM_194279 issue) can be downloaded here [Makefile].
The long implementation with two lines in the makefile uncommented (NUCDB=db/nt.01 and PROTDB=db/nr.01) contain three processes that are speed limiting and single threaded (blastx NG_008953b: NG_000006b: NG_007385b). These three runs delay the total finishing time by a high percentage.
To run the command under Windows on 32 threads using Cygwin: ./timethis "make -j 32 all"
Table: Benchmark queries in BLAST Benchmark 2013 (short version), tblastx is speed limiting. A sequential runtime would last 6 minutes. Because tblastx is speed-limiting, the total runtime is always determined by the tblastx benchmark time. On a 32-thread machine the parallel total runtime is 2:15 min.
| Num | Name | queries | time [min] |
| 1 | blastn | 100 | 00:32.9 |
| 2 | blastp | 40 | 00:13.9 |
| 3 | blastx | 24 | 00:06.5 |
| 4 | megablast | 24 | 00:55.5 |
| 5 | tblastn | 10 | 00:20.1 |
| 6 | tblastx | 10 | 02:00.9 |
| 7 | idx_megablast | NA | 01:51.4 |
| Total time (sequential) [min] | 06:01.2 | ||
| Total time (parallel) [min] | 02:15.1 | ||
Download
The benchmark files (just extract to a folder)
ftp://ftp.ncbi.nih.gov/blast/demo/benchmark [PPT] [ZIP]
The blast files for WIN32, LINUX, Solaris, IRIX, Sparc, PPC, MacOSX
ftp://ftp.ncbi.nih.gov/blast/executables/LATEST
Windows Timethis
TimeThis.exe
Make.exe for running the benchmark from MingW - with all UNIX tools for Windows
MingW or [EXE]
Adjust the path folders in the makefile so they point to the right benchmark files and executables (put them in one root folder)
Run the benchmark with the make command: timethis "make -j 4 >> bresult.txt"
This will run the benchmark with four threads provided by make (it will split the workload).
The argument -a "Number of processors to use" is not working in version 2.2.13 so you have to use make -n