QlikView will use as much memory available as it needs, up to the Working set Low as specified in the QMC > System > Setup > QlikView Servers > Performance (do not change those settings unless you are positive you have to change them).
This is expected and it is the strength of QlikView: the more contents in memory the faster response times the users will get, the better experience.
In short and oversimplifying, opening the document uses some memory, and if the document is set to Preload, it will be open automatically every time the QlikView Server service restarts. This way, the first user to connect will have the document already in memory. Next, every time a user clicks, the results of the calculations are cached into memory, so the next user to click in the same place will get the calculations from cache, instead of aggregating again.
After some use, it is normal that QlikView will be eventually using all the memory it has available, even if the users sessions timed out or disconnected. If the server is properly sized and QlikView correctly set up, this will not cause any harm in performance or to the operating system.
For more details and deep dive on all things Qlik performance and resource usage, check the documents in the Qlik Scalability group.