Skip to main content
Logo image

Section 2.5 Getting math right

Java’s two numeric types, int and double, serve different purposes: if we’re counting things ints are great. And doubles are great for scientific calculations such as those involving measured quantities. But some computations will require us to work with values of both types which will require us to control when we convert between the two types if we want to get mathematically correct results.
Also both data types have some limitations that we need to understand in order to avoid mathematically nonsensical results. (For instance, you’d probably get an F in math class if you said 2147483647 + 1 was -2147483648. But that’s what you’d get from Java due to the limitation on the range of values that can be represented with an int!
In this lesson we will look at the cast operator that we can use to convert between types, a recipe for rounding doubles to their nearest int equivalent, and those limitations on both int and double values.

Subsection 2.5.1 Casting

In Java, type casting is used to convert values from one type to another. By casting we don’t mean something to do with fishing, but it is a similar idea to casting a bronze, without needing to heat anything to 913 degrees Celsius. But like molten bronze is reshaped by melting it and pouring it into a mold, our data is reshaped via a cast operator. In Java when you cast you are changing the β€œshape” (or type) of the value.
Figure 2.5.1. Bronze casting changes the shape of the metal.
The cast operator, which looks like (int) and (double) placed before any expression (a literal, a number, a variable, or more complex expression in parentheses) produces a value of the given type by converting the value of the original expression to the new type. The casting operators (int) and (double) can be used to convert from a double value to an int value (or vice versa).
For example, (double) 1 / 3 will evaluate to a double value 0.33333333 instead of an int truncated value 0. And (int) 3.6 will evaluate to the int value 3, chopping off the digits to the right of the decimal point.
A conversion from int to double is called a widening conversion because a double can represent any int value but not vice versa; thus a double is considered a wider data type than an int.

Note 2.5.2.

ints in Java are always 32-bit signed values which mean they can represent values from \(-2^{31}\) to \(2^{31} - 1\text{,}\) inclusive, while the range of consecutive integer values that can be represented by a double is from \(-2^{53}\) to \(2^{53}\text{,}\) inclusive. (A double can also represent much larger values but with limited precision.) You can refer to the minimum and maximum int values with the constants Integer.MIN_VALUE and Integer.MAX_VALUE.

Activity 2.5.1. Dividing mixed types.

Run this code to see how Java handles division and what casting can do to the results. Notice what happens when you divide an int by an int or an int by a double or an int cast to a double divided by an int.
Then add another line that divides 5 by 2 but casting one of the values to (double). What is the result?

Subsection 2.5.2 Casting to double

As you can see in the exercise above, when Java divides two ints, it produces an int result by truncating the actual mathematical result, removing anything after the decimal point. Thus 3 / 4 evaluates to 0, not 0.75 as we might expect. It also does not evaluate to 1 even though 0.75 is closer to 1 than it is to 0; truncating is not the same as rounding!
But as we learned in SectionΒ 2.3Β Arithmetic expressions, in any expression involving a double, the double is β€œcontagious” and will cause the other operand to be turned into a double and thus the value of the expression as a whole to be computed using the rules of double math. Thus the expression 3.0 / 4 is evaluated as if it had been written 3.0 / 4.0 and produces the double value 0.75.
If an expressions involves at least one literal value then we can just use a literal double to force Java to use double math. But what if we want to evaluate an expression using int variables but under the rules of double math?
That’s where casting comes in. Since casting an int to double produces a double value, if that value is the operand in an expression, it will then infect the other operand and cause the expression as a whole to produce a double. This is especially useful when we have int variables that we want to do non-integer division with:
int total; // a variable containing the sum of a bunch of ints
int count; // the number of ints that went into total

// Compute the average of the bunch of ints summed into total.
double average = (double) total / count;
Another thing to notice is that cast is an operator, just like +, *, -, /, and %. So it has a precedence just like all the other operators and it is actually quite high. The expression above works because the cast of total to double happens first and then that double value infects count and the whole expression is computed using double math. If instead, we had written:
double average = (double) (total / count);
then the division would have happened first, yielding a truncated int value which would thenβ€”too lateβ€”be converted to the equivalent double value, losing the fractional part of the true mathematical answer.

Subsection 2.5.3 Casting to int

Since casting a double value to int throws away mathematical precision it might seem that we’d rarely want to do that. But we can use truncation, with a bit of arithmetic, to round double values to the nearest int value by adding or subtracting 0.5 and then casting the result with (int):
double number;    // positive value from somewhere
double negNumber; // negative value from somewhere

int nearestInt = (int) (number + 0.5);
int nearestNegInt = (int) (negNumber – 0.5);
For example, if we divide 7.0 / 4.0 we get 1.75. And if we cast that to an int, it would be truncated to 1. But if we want to round it to the nearest int, rather than truncating it, adding 0.5 will produce a number that is greater than 2.0 and thus will get truncated to 2. Assuming we’re okay rounding above the next integer value if the decimal part is greater than 0.5, as it is here. Then casting that value to an int will truncate down. So in this case 1.75 + 0.5 gives us 2.25 which is then truncated to 2. On the other hand adding 0.5 to the result of evaluating 5.0 / 4.2, namely 1.25, only gets us to 1.75 which truncates back to 1 which is the nearest integer to 1.25.
And notice that here we need to parenthesize the expression number + 0.5 so that it is evaluated first and then the result cast to a double. Without the parentheses, the cast to int would convert number to an int first because of the higher precedence of casting than division, and then that truncated int value would be infected by the 0.5 giving us a double result with a fractional part of 0.5.

Activity 2.5.2. Test rounding.

Run the code below to see how the formula of adding or subtracting .5 and then casting with (int) rounds a positive or negative double number to the closest int. Add a line of code that rounds number + 2.3 to the nearest int.

Subsection 2.5.4 Limits of int and doubles

Both the int and double data types have limitations in the numbers they can represent. These limitations are due to the fact that for efficiency reasons computer hardware represents both types using a fixed number of bits, 32 bits for each int and 64 bits for a double.
For ints, this means there is a biggest number that can be represented, just like there’s a biggest number that can be represented in a fixed number of digits. Because ints can represent both positive and negative numbers, half of the bit patters are used to represent negative numbers and half to represent the non-negative numbers including zero. The maximum int value is \(2,147,483,647\) also known as \(2^{31} - 1\text{.}\) The minimum value (most negative) is \(-2,147,483,648\) or \(-(2^{31})\text{.}\) There are constants in Java, Integer.MAX_VALUE and Integer.MIN_VALUE, that we can use if we need to refer to those values in a program.
If we try to store value smaller than Integer.MIN_VALUE or bigger than Integer.MAX_VALUE in an int variable or if an expression that is supposed to produce an int value would produce a value outside that range, the result overflows and produces a mathematically incorrect value. Roughly speaking the value will β€œwrap around” so, for instance, Integer.MAX_VALUE + 1 gives us the same value as Integer.MIN_VALUE and Integer.MIN_VALUE - 1 brings us back to Integer.MAX_VALUE.
If we try to write a literal int that is out of range the compiler will give us an error as shown in the exercise below.

Activity 2.5.3.

Try the code below to see that there are limits to how big int values can be. Fix the code by deleting the last 0 in the numbers to make the values small enough to be stored in an int.
The double type, despite using twice as many bits per number as ints, also has limitations. Doubles work more like scientific notation, storing a signed magnitude, and an exponent. This means they can represent both very small numbers (the smallest positive value is approximately \(4.9 \times 10^{-324}\)) and very large (the largest positive double value is approximately \(1.8 \times 10^{308}\text{.}\))
However this means, that like scientific notation, there is a limit to the precision with which doubles can represent numbers, about 14-15 decimal places. That has a few consequences. One is that irrational numbers like \(\pi\) and \(\sqrt{2}\) that have an infinite decimal representation can only be approximated. But also simpler numbers like 1/3 which have repeating decimal representations can only be approximated. So the value we get if we evaluate 1.0 / 3.0 is actually roughly \(1.85 \times 10^{-17}\) less than 1/3.
In practice, of course, 14-15 digits of precision is plenty for most purposes. But you should be aware that the accuracy of any calculation on a computer is limited by the fact that computers can only hold a limited number of digits. When these limitations lead to incorrect results it is called a round-off error. Programmers who write software that does complex scientific calculations do something called numeric analysis to make sure their programs won’t be affected by round-off errors but most of us don’t have to worry about that most of the time.

Note 2.5.3.

Although it’s not on the AP exam, you can format long decimal numbers to just show 2 digits after the decimal point with the following code using printf a formatted print method or format instead of println. It takes a format string like %.02f which tells printf to print a floating point number indicated by the % with 2 digits after the decimal point. See https://docs.oracle.com/javase/tutorial/java/data/numberformat.html for more information. You can also use escape characters like \n in the format string to do a newline. Try it below.

Activity 2.5.4. Using printf.

Run the code below to see how a decimal number can be formatted to show 2 digits after the decimal point. Try it with 2.0/3.

Activity 2.5.5. Integer division.

True or false: Java rounds up automatically when you do integer division.
  • Did you try this out in Active Code? Does it work that way?
  • false
  • Java throws away any values after the decimal point if you do integer division. It does not round up automatically.

Activity 2.5.6. Casting.

True or false: casting always results in a double type.
  • Try casting to int instead of double. What does that do?
  • false
  • Casting results in the type that you cast to. However, if you can’t really cast the value to the specified type then you will get an error.

Subsection 2.5.5 Coding Challenge: Average three numbers

This would be a good project to work together in pairs, and switch drivers (who has control of the keyboard in pair programming) after every line of code. In the code below, type in three made up int grades and then sum and average them. Use casting to report the result as a double. For example, if the grades are 90, 100, and 94, the sum of the three numbers is 90 + 100 + 94 = 284, and the average is the sum 284 divided by 3 which, when cast to a double, is 94.666667. You should use your variables instead of the numbers in your formulas. Follow the pseudo code below.

Project 2.5.7. Averaging three numbers.

Type in three made up int grades and then sum and average them. Use type casting to report the result as a double. If you do this challenge on replit.com (see template and links below), please paste your repl link here to turn it in.
Your teacher may suggest that you use a different Java IDE for this challenge so that you can use input to get these values using the Scanner class, for example with this JuiceMind template or replit template that you can use if you want to try the challenge with input.

Subsection 2.5.6 Summary

  • Type casting is used to convert a value from one type to another.
  • (AP 1.5.A.1) The casting operators (int) and (double) can be used to convert from a double value to an int value (or vice versa).
  • (AP 1.5.A.2) Casting a double value to an int causes the digits to the right of the decimal point to be truncated (cut off and thrown away).
  • (AP 1.5.A.3) Some code causes int values to be automatically cast (widened) to double values. In expressions involving doubles, the double values are contagious, causing ints in the expression to be automatically converted (β€œwidened”) to the equivalent double value so the result of the expression can be computed as a double.
  • (AP 1.5.A.4) Values of type double can be rounded to the nearest integer by (int)(x + 0.5) or (int)(x – 0.5) for negative numbers.
  • (AP 1.5.B.1) The constant Integer.MAX_VALUE holds the value of the largest possible int value. The constant Integer.MIN_VALUE holds the value of the smallest possible int value.
  • (AP 1.5.B.2) Integer values in Java are represented by values of type int, which are stored using a finite amount (4 bytes) of memory. Therefore, an int value must be in the range from Integer.MIN_VALUE to Integer.MAX_VALUE, inclusive.
  • (AP 1.5.B.3) If an expression would evaluate to an int value outside of the allowed range, an integer overflow occurs. The result is an int value in the allowed range but not necessarily the value expected.
  • (AP 1.5.C.1) Computers allot a specified amount of memory to store data based on the data type. If an expression would evaluate to a double that is more precise than can be stored in the allotted amount of memory, a round-off error occurs. The result will be rounded to the representable value. To avoid rounding errors that naturally occur, use int values or round doubles to the precision needed.

Subsection 2.5.7 AP Practice

Activity 2.5.8.

Which of the following always returns the correct average (to 14-15 decimal places) if total is an int variable that is the sum of three values?
  • (double) (total / 3);
  • This does integer division before casting the result to double so it loses the fractional part.
  • total / 3;
  • When you divide an integer by an integer you get an integer result and lose the fractional part.
  • (double) total / 3;
  • This will convert total to a double value and then divide by 3 to return a double result.
  • (int) total / 3;
  • This will return an int value losing the fractional part.
You have attempted of activities on this page.