Better (non Fatal) Out of memory handling

Post your feature requests here. Note that we cannot promise that any of these posts will be commented on or that requested features will be implemented.
FSMP
Posts: 678
Joined: Sat Sep 25, 2010 9:38 am

Better (non Fatal) Out of memory handling

Postby FSMP » Thu Jan 28, 2016 2:27 pm

Surely, something can be done to improve P3D's behavior when it starts to run out of memory. ??

Just "Giving up" and terminating the application with the pitiful "Out of memory" message, is really now unacceptable in a simulator of this class, and appears to be an inherited act of desperation, from the early FSX development days, when it was never really expected that FSX would become so loaded with add-ons, that memory would be an issue.

It's almost as if back then, this handling was only coded to provide "very basic error handling" for failed memory allocation request, which were only really expected to happen if there was some unintentional memory leak ??

Isn't it about time to address this crude & desperate action, and introduce some better memory management, that can monitor memory usage, and if still needed, release some of the allocated memory that is being used up by non-essential resources (ie High slider settings), and in real time, sacrifice some performance/resolution/etc, and keep the simulator running, rather than continue running, and actually reaching the point where a final memory request demand fails and causes the simulator to close down.

User avatar
Snave
Posts: 1004
Joined: Mon Nov 25, 2013 6:01 pm

Re: Better (non Fatal) Out of memory handling

Postby Snave » Thu Jan 28, 2016 10:38 pm

The simulator does not have an OOM problem

YOUR simulator with addons does.

Where to go for the solution? L-M?

Think not.
No door is closed to an open mind.
Cats open doors...

simmerhead
Posts: 107
Joined: Sun Sep 25, 2011 12:25 pm

Re: Better (non Fatal) Out of memory handling

Postby simmerhead » Mon Feb 01, 2016 10:58 am

OOM is most likely caused by a poorly designed/optimized addon or a number of them on top of eachother.

That said, it would have been nice if it was somehow possible to improve memory management by making the simulator start dumping data from VAS when reaching the ~4GB limit, instead of getting the dreaded OOM message and resulting closing of the sim.

User avatar
WarpD
Posts: 1451
Joined: Mon Feb 14, 2011 5:29 am

Re: Better (non Fatal) Out of memory handling

Postby WarpD » Mon Feb 01, 2016 12:55 pm

I think on one hand it would be nice if when the software goes to allocate memory and there isn't any left that it would simply exit whatever process it was doing gracefully and allow the sim to continue to function. Unfortunately as I sit here and think about it... there's no safe way to do this. What if the allocation had to do with something more important than simply loading a texture? The sim would probably crash horribly. That's not a good thing either.

In the long run, a 64-bit application will allow us to run more complex sceneries with fewer memory concerns. In the short term, it's going to be up to the end-user to intelligently set up boundaries with their scenery installations so that they don't exceed the limitations that do exist.
Ed Wilson
Senior Developer
Mindstar Aviation

User avatar
Mark Adams
Posts: 63
Joined: Thu Dec 18, 2014 2:12 pm

Re: Better (non Fatal) Out of memory handling

Postby Mark Adams » Mon Feb 01, 2016 3:09 pm

+1 on 3rd party developers definately need to play their part by optimising for performance.

The original post was asking about handling rather than cause which seems valid when we don't get to pick and choose what add-ons get installed.

Obviously the platform and OS have to protect themselves but I'd be interested to know if there is any wiggle room to be had.

ananda
Posts: 440
Joined: Tue Nov 26, 2013 11:40 am
Location: Harton Village, UK

Re: Better (non Fatal) Out of memory handling

Postby ananda » Mon Feb 01, 2016 4:00 pm

Mark Adams wrote:The original post was asking about handling rather than cause which seems valid when we don't get to pick and choose what add-ons get installed.

You have absolute control over which addons are installed.

User avatar
Mark Adams
Posts: 63
Joined: Thu Dec 18, 2014 2:12 pm

Re: Better (non Fatal) Out of memory handling

Postby Mark Adams » Mon Feb 01, 2016 5:03 pm

Unfortunately I do not

I (we) provide physics/systems/simobject content to aircraft manufacturers and for military and certified training simulators/PTT etc; I don't own the hardware or set the requirement.

The system may require hugely complex environments and systems or have to operate the real flight surfaces/servos on a tethered experimental aircraft - all kinds of weird things that must go in or nobody gets paid.

FSMP
Posts: 678
Joined: Sat Sep 25, 2010 9:38 am

Re: Better (non Fatal) Out of memory handling

Postby FSMP » Mon Feb 01, 2016 11:22 pm

FSMP wrote: <-------- and introduce some better memory management, that can monitor memory usage, and if still needed, release some of the allocated memory that is being used up by non-essential resources (ie High slider settings), and in real time, sacrifice some performance/resolution/etc, and keep the simulator running, rather than continue running, and actually reaching the point where a final memory request demand fails and causes the simulator to close down. ------>


If you are going to contribute to this thread, it might help if you read the OP's post a little more carefully ... :P

Telling me what "MY Simulator" is, when you do not have a clue what I have, is not really that constructive ?

User avatar
WarpD
Posts: 1451
Joined: Mon Feb 14, 2011 5:29 am

Re: Better (non Fatal) Out of memory handling

Postby WarpD » Tue Feb 02, 2016 3:15 am

Ok I've given this more thought and it's not as simple as you think.

When an application requests memory, it might be only 128 bytes of memory... but the system doesn't do allocations that small. It grabs it in much larger blocks, setting them aside and then pulling out pieces as they're requested by the application. You don't get an OOM for requesting 128 bytes... you get an OOM because the system goes to allocate a new reserved block and there's not enough memory to accomplish that.

So... just free some up, right? Well, technically you don't do that either. You tell the system you're done with it, but then memory can be fragmented just like your hard drive... and it doesn't always get reorganized. Your application never sees any of this. It can look at the amount of available memory and see that it has enough memory to allocate 128 bytes... but still receive an OOM because the system needs to obtain a new larger block and one isn't available.

The absolute best defense against an OOM is for developers to put more effort into not eating up memory. I'll admit that in the past I was rather lax about memory usage... after all, who'd ever use up 3gig... right?

In the long run, it's going to be up to the third party developers to better manage as there's only so much L-M can do and they're doing a pretty decent job so far. In time 64-bit will give you a small amount of breathing room... but then developers will get sloppy again... and you'll get OOM messages because your video memory ran out.
Ed Wilson
Senior Developer
Mindstar Aviation

FSMP
Posts: 678
Joined: Sat Sep 25, 2010 9:38 am

Re: Better (non Fatal) Out of memory handling

Postby FSMP » Thu Feb 04, 2016 11:11 pm

WarpD wrote:Ok I've given this more thought and it's not as simple as you think.

When an application requests memory, it might be only 128 bytes of memory... but the system doesn't do allocations that small. It grabs it in much larger blocks, setting them aside and then pulling out pieces as they're requested by the application. You don't get an OOM for requesting 128 bytes... you get an OOM because the system goes to allocate a new reserved block and there's not enough memory to accomplish that.

So... just free some up, right? Well, technically you don't do that either. You tell the system you're done with it, but then memory can be fragmented just like your hard drive... and it doesn't always get reorganized. Your application never sees any of this. It can look at the amount of available memory and see that it has enough memory to allocate 128 bytes... but still receive an OOM because the system needs to obtain a new larger block and one isn't available.

The absolute best defense against an OOM is for developers to put more effort into not eating up memory. I'll admit that in the past I was rather lax about memory usage... after all, who'd ever use up 3gig... right?

In the long run, it's going to be up to the third party developers to better manage as there's only so much L-M can do and they're doing a pretty decent job so far. In time 64-bit will give you a small amount of breathing room... but then developers will get sloppy again... and you'll get OOM messages because your video memory ran out.


You are, of course, 100% correct in your explanation/analysis - obviously a programmer, as am I.

I never said it was SIMPLE, but knowing now, in hindsight after FSX was released, that running out of "contiguous" memory blocks, is an issue, a P3D specific "Memory Management System" within P3D would improve things.

Knowing the root cause of the OOM issues, could lead to the design of an appropriate "Memory Management System", to far better handle memory requests and memory allocation.

User avatar
WarpD
Posts: 1451
Joined: Mon Feb 14, 2011 5:29 am

Re: Better (non Fatal) Out of memory handling

Postby WarpD » Fri Feb 05, 2016 2:28 am

However, it's only an issue when you add third-party products into the mix... and they're most likely going to use the default OS memory management system, which means the core sim still has no control.

I really don't see how you can force third-party products to behave with regards to memory. Since they're the primary source of OOM issues... and the secondary source is the user who wants to park a Hummer in a space large enough for a Yugo (loading up on tons of 4kx4k textures and everything else maxed out, etc). Neither can be controlled very well because they're going to do what they want to do.

L-M did something in the current Prepar3D version to reduce OOMs... they limted LOD settings. People aren't happy with that... and it's definitely a major source of OOMs.
Ed Wilson
Senior Developer
Mindstar Aviation

User avatar
WarpD
Posts: 1451
Joined: Mon Feb 14, 2011 5:29 am

Re: Better (non Fatal) Out of memory handling

Postby WarpD » Sat Feb 06, 2016 4:17 am

So... discussing this further... for L-M to implement their own memory management system that eliminates the OS... they'd have to allocate maximum memory at program start so that the OS gives them a single large block... and then work within that block. Is there really any other way?
Ed Wilson
Senior Developer
Mindstar Aviation


Return to “Prepar3D Feature Requests”

Who is online

Users browsing this forum: No registered users and 7 guests