Monday, April 18, 2011

Microsoft(tm) Calculator Challenge

The Microsoft Calculator Challenge

Here is challenge for those of you with some time on your hands (it has happened to me) and illustrates an interesting testing technique.

Background

I have been a nemesis to the calculator people at Microsoft.  One time in the late 90's, I had some time on my hands (See?  I told you.) while I was working, and I was supposed to be testing but the system under test was not available (probly never happened to you) and I was looking for something to test.

My desktop at work was a PC running Windows NT and there it was: Windows Calculator.

So I began trying things.  It was definitely broken.  Mostly precision issue matters.  For a complete list, see my Ph.D. dissertation, "Appendix A - Calculator Anomalies."

Microsoft "fixed" most of these problems by adding "infinite" precision.  You can try this with the following string (without quotes) pasted into the scientific view (ALT-2) of MS Calculator: "1x9999s".  You can click on "Continue" as many times as you want. 

So I faced that challenge myself, since what I considered a really stupid fix made me somewhat unhappy with Microsoft.  I had developed a random test case generator and it generated a very long string that I could paste into calculator as above and got a catastrophic failure.  I spent some time reducing that test case and found that the following string (again, without quotes) would cause that same symptom:

"(((((((((((((((0=)))))))))))))))"

During the test string reduction, I also discovered the following string that produced an interesting result:

"(0=)(2+2=)"

The problems with these test cases persisted in subsequent versions of Windows including 2000, XP, and Vista.  If you have one of those operating systems, try those test cases.

However, if you have Windows 7, as I now have, these cases no longer "fail."  Microsoft has once again "fixed" the problem by terminating the operation once the "=" sign is entered.  Thus:

"(((((((((((((((0="

Is a complete and valid calculator operation.

The Challenge

I am incensed!  So you know what I did?  I removed the "=" from my random test case generator and generated another long string, and guess what?  It still fails catastrophically in Windows 7.  (I haven't tried it with other Windows versions, but I'll bet it fails.)

The original test case is  17,475 characters long.  As before, I have reduced this case to a case that is only 376 characters long as follows:

(1+(1+(1/(1/((1+((1+1)+1))+(1+1/((1+(1+(1+(1+(1+(1+(1+(1+(1+((1/(1+(1/(1/((1+(1+(1/((1+(1/(1)+(1/(1+(1+(((1))/1))/1))))/1))))))/1)))+(1+(1+(1+(1/(1+(1)/1))/1))+1)/1)/1)/1))+1))+1)/1)/1+(1/1)))))/1))/1)/1+(1+(1+((1/(1+((1/(1/(1/(1+(1/(1/(1+(1+((1+1)+1)/1)+(1+(1/(1+(1/(1+(1+1+1)+1)/1))/1)+1)))+1/(1/(1)/(1/((1)/1)/1)/1)+1)))/1)/1)/1)))/1)/1)+(1/(1/(1+((1+(1)/1)/1))))/1)/1)/1)/1

So the challenge is this:  What is the minimum length string that will cause this failure?

An interesting note: If you remove that last "/1" from the test case, it will not fail.

Hint:  I save the string in MS Notepad, make a change reducing the length, copy it, and paste it into calculator and if it fails I save it under a new name, if not, I restore the previously failed version.  The reduced case you see above is TC18.txt.  Be sure there are no line breaks in the string if you copy it from here.

I am also interested in intermediate results, different results, different symptoms, and failing results with a different OS or application.

4 comments:

  1. So I gave up waiting to see if someone would solve this. So I have found:

    TC92: "+1*(((((((((((((((((((((((1+((1))))))))))))))))))))))))))-"

    Improve on that!

    ReplyDelete
  2. As of late October, 2015, this test case still fails on Windows 10. Have I mentioned that I tried installing Windows 10 here, and it erased my hard drive? Fortunately, apparently they save your old installation on the cloud because after a tedious search for a recovery method, I was able to reinstall this Windows 7, still with my applications.

    ReplyDelete
  3. Using Windows calculator version 10.1705.1301.0, the calculator no longer accepts any input starting or ending in a mathematical operator, meaning the test case posted on May 5, 2012 simply returns a generic "invalid input" message. The 376 character string likewise causes an invalid input message when entered into the calculator (at least when pasting, I haven't tried entering manually)

    Embedded within the 376 character string I found at least two other strings which cause two different failures of the calculator. The first string causes the calculator to crash when pasted in and stands at 187 characters:

    (1+(1+(1/(1/((1+((1+1)+1))+(1+1/((1+(1+(1+(1+(1+(1+(1+(1+(1+((1/(1+(1/(1/((1+(1+(1/((1+(1/(1)+(1/(1+(1+(((1))/1))/1))))/1))))))/1)))+(1+(1+(1+(1/(1+(1)/1))/1))+1)/1)/1)/1))+1))+1)/1)/1+(1

    The second string provides a non-integer value, 8.142857 repeating, and stands at 133 characters:

    (1+(1+(1/(1/((1+((1+1)+1))+(1+1/((1+(1+(1+(1+(1+(1+(1+(1+(1+((1/(1+(1/(1/((1+(1+(1/((1+(1/(1)+(1/(1+(1+(((1))/1))/1))))/1))))))/1)))

    -QB

    ReplyDelete
  4. QB,

    Thanks for your entry and the update.

    I am using Windows 7 old enough to not show the version number.

    You first example does crash my version of calc.exe.

    I would expect your second version to produce a real number because it has a division by an expression that can be a sum and be other that one.

    My last test case was tc92 which means that I have 91 other test cases that cause calc to fail and they may not all have the same cause. They all have the property that calc crashes.

    I have a theory that suggests that the failure that causes this symptom may be exploitable and I am looking into that.

    Both of our examples crash with an unserviced exception in NTDLL. So the question is now: "What causes the exception?"

    Thank you for your interest.

    Alan (Softtest123)

    ReplyDelete