The open box will have the following dimensions: l=6-2x,w=6-2x,h=x
(You take a segment of length x from both corners on a side.)
The volume is lxwxh so:
To find the maximal volume we take the first derivative and find the critical points by setting equal to zero:
The critical points are x=3 or x=1. From the problem, x=3 will not work as you will have no material left. Therefore x=1 is the only possible solution.
The volume will be V=(4)(4)(1)=16cu units.