HomeSearch

Python Timeit, Repeat Examples

Use the timeit module to benchmark code. Call timeit, repeat, multiple statements and methods.
Timeit. Micro-benchmarks are not always useful. But they are sometimes fun. Which code unit executes faster in a Python program? We answer this question with timeit, a module that benchmarks code fragments.
Example. We introduce timeit with a simple example. We must import timeit with the "import timeit" statement. This is required unless you use the command-line syntax. Here we time two string-creation expressions.

And: We pass the statements in quoted strings to the timeit.timeit method. We increase the iterations by specifying a number argument.

Note: The numbers here are too close to know for sure which is faster. We can use the repeat() method to receive better information.

Python program that uses timeit import timeit # The instructions being timed. print('y' * 3) print('y' + 'y' + 'y') # Call timeit on the statements and print the time returned. # ... Specify optional number of iterations. print(timeit.timeit("x = 'y' * 3", number=10000000)) print(timeit.timeit("x = 'y' + 'y' + 'y'", number=10000000)) Output yyy yyy 0.2625868763293428 0.26622904456542135
Repeat. Repeat is the same as timeit except it benchmarks repeatedly: it calls timeit internally several times. The default repetition is 3. We can increase or decrease this by specifying the repeat argument.

Here: We increase the number of iterations of the string-multiplying code shown in the previous example. We start to get repeatable data.

And: It seems to indicate that adding three strings together is faster than multiplying one by 3.

Python program that uses repeat import timeit # Call repeat. print(timeit.repeat("x = 'y' * 3", number=100000000, repeat=3)) print(timeit.repeat("x = 'y' + 'y' + 'y'", number=100000000, repeat=3)) Output [2.7390200865497176, 2.7475431168207223, 2.7429300279022177] [2.6369100279087014, 2.631240758828813, 2.6300020650299665]
Command-line. Some programmers extensively use the command-line. The timeit module can be invoked directly from the command-line. This avoids creating an entire new program file. Timeit returns usec (microseconds) in the output.

Tip: You will need to be careful with quotation marks when using the command-line. You may need to escape them, depending on your system.

Command line for timeit: Windows 10 C:\Users\Sam>C:\Python33\python.exe -m timeit "x = \"y\" * 3" 10000000 loops, best of 3: 0.0273 usec per loop
Multiple statements. With timeit, we can use multiple statements. We separate them with a semicolon. This is not typical of Python syntax, but it works. This makes it easier to specify longer (but not really long) code fragments.

Note: For longer code fragments, please use the setup argument and call a method. The next example demonstrates.

Python program that uses timeit, semicolon import timeit # Use semicolon for multiple statements. print(timeit.repeat("a = 2; a *= 2", number=100000000)) print(timeit.repeat("a = 1; a *= 4", number=100000000)) Output [7.334341642836696, 7.333336683198793, 7.332224095625474] [7.235993375046725, 7.247406798908553, 7.256258872415835]
Methods, setup. Let us continue with methods. We can benchmark custom methods in timeit by specifying a setup argument. In this argument, please specify an import statement that indicates the methods you invoke.

Here: We benchmark the a() method against the b() method. As expected, the a() method is faster. It does less.

Python program that uses timeit, methods, set up import timeit def a(): return 1 def b(): return sum([-1, 0, 1, 1]) # Test methods. print(a()) print(b()) # Pass setup argument to call methods. print(timeit.repeat("a()", setup="from __main__ import a")) print(timeit.repeat("b()", setup="from __main__ import b")) Output 1 1 [0.11886792269331777, 0.11894442929800975, 0.11940800745355873] [0.5983422704501993, 0.6003713163771788, 0.6014057764431624]
Discussion. Benchmarks are fraught with problems. It is hard, if not impossible, to isolate external factors. And often we should not want to: no programs run without external interference in the real world.
In testing timeit, I found it too has many problems. Its syntax is clunky. After all, what sort of Python program separates statements with semicolons? And the syntax for invoking methods is cumbersome.

Further: I found that the ordering of calls to timeit impacts the results. This makes it harder to trust the results of timeit.

In many ways, I like timeit. It provides a standardized way to perform these benchmarks. And all the alternatives too have problems. Overall, focusing on higher-level concerns and code quality is a better use of your time.Timeit: Python.org

And: With the new JIT compilation technologies, such as PyPy, micro-benchmarks must be updated. Performance vastly changes.

JIT Tests
Summary. It usually costs more time running micro-benchmarks that you will get back in increased speed. But these benchmarks are helpful for learning the performance characteristics of a language. With timeit, we have another benchmarking option.
© 2007-2019 Sam Allen. Every person is special and unique. Send bug reports to info@dotnetperls.com.
Home
Dot Net Perls