C++ 28 Jan 2011 16:21:01
C++ Read Whole File Performance
A performance comparison of the speed of various ways to read an entire file into an std::string in C++.
Idea from BD-Calvin in #C++ on QuakeNet.
Sources
- Ticks counted via cycle.h (local mirror)
- Source: speed-read-whole-file.cpp
- File used was generated with dd if=/dev/urandom of=random100 bs=1M count=100
Things Tested
- writing to a stringstream, then pulling data out as string
- constructing a string via streambuf_iterator
- getting file size via seeking, then preallocating a string to read into
- getting file size via stat(), then ditto…
Observations
- During the test, the seek method was twice as slow as the stat() method, until the file was buffered by the OS. For smaller files this would make no difference, but if one plans on reading larger files then stat() is better.
Linux: GNU g++ 4.4.1
- Compiler: GNU g++ 4.4.1 -O3
- Arch: Linux 2.6.27 x86_64, 2.66GHz Xeon, 8 GiB RAM
| g++ 4.4.1 | Ticks | Factor | Allocations | Bytes Allocated |
|---|---|---|---|---|
| Stat | 438702544 | 1.00 | 5 | 104870440 |
| Seek | 439898520 | 1.00 | 5 | 104870440 |
| Stringstream | 1063113392 | 2.42 | 23 | 373300883 |
| Iterator | 2089106752 | 4.76 | 24 | 268439653 |
Linux: LLVM clang++
- Compiler: clang++ 2.9 (trunk 124366) -O3
- Arch: Linux 2.6.27 x86_64, 2.66GHz Xeon, 8 GiB RAM
| clang++ 2.9 (trunk 124366) | Ticks | Factor | Allocations | Bytes Allocated |
|---|---|---|---|---|
| Stat | 438471072 | 1.00 | 5 | 104870440 |
| Seek | 438515248 | 1.00 | 5 | 104870440 |
| Stringstream | 1068628728 | 2.43 | 23 | 373300883 |
| Iterator | 2598286608 | 5.93 | 24 | 268439653 |
Windows: MSVC++ 2010
- Compiler: MSVC++ 2010 _SECURE_SCL=0
- Arch: Windows 7 64 bit, 1.60GHz Core i7 Q720, 8 GiB RAM
| VC++ 2010 | Ticks | Factor |
|---|---|---|
| Stat | 418779260 | 1.00 |
| Seek | 418555987 | 1.00 |
| Stringstream | 4828526918 | 11.53 |
| Iterator | 4845144025 | 11.57 |
