P3D v5.3 HF2 | No haze/atmo (again)

Any issues, problems or troubleshooting topics related to the Prepar3D client application.
disco79stu
Posts: 57
Joined: Thu Jul 16, 2020 9:05 pm

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by disco79stu »

The haze is indeed back with the above hack, but now I have a strange, very prominent grey line at or above the horizon when at cruise level.
That is gone with AS max visibility set to 200 (wich I don‘t really prefer because it makes the sky too poppy).

So it‘s really a matter of which is the lesser of several evils, I do not enjoy this constant tweaking and fixing and bug-accepting…. With every version they fix 5 bugs and introduce 3 new ones (or bring back old ones). The fact that we always need several hotfixes for every v5 version now is somehow worrying.
Wise87
Posts: 19
Joined: Thu Dec 15, 2016 5:06 am

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by Wise87 »

I hope LM provides a replacement file so we don't have to wait for a HotFix 3. I don't mind making the mentioned tweak but would like confirmation by LM that the mentioned tweak is not going to cause other issues. If so then I would like them to provide the correct entries to make like they did before.
User avatar
Sabre57
Posts: 493
Joined: Thu Jan 23, 2014 12:45 am

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by Sabre57 »

This is strange because I'm getting haze even when AS is set at 200 miles max visibility.



https://i.imgur.com/lutiCro.jpg
I9-9900
32GB RAM
RTX 3090 FTW
Assorted SSDs and HDDs
3x55" TVs
Alpha/Bravo Honeycombs
SPAD.Next, ASV6
Mejjo
Posts: 19
Joined: Thu Aug 24, 2017 1:17 pm

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by Mejjo »

disco79stu wrote: Fri Jan 21, 2022 7:08 pm
hotbso wrote: Fri Jan 21, 2022 1:26 pm HF1 seemed to "haze away" stars at night so they fixed this in HF2. Unfortunately that removed haze at day time as well (at least when flying high or whatsoever). The following fix (actually it's a hack) blends the new and the old behavior so you have stars at night and haze at day.

Change ShadersHLSL\FuncLibrary.fxh at around line 512. It should look like this

Image


Clear shaders and that's it. Don't blame me for problems.
Wow, thanks man!
Does the trick for me (even better with Envshade activated).
Still hoping for an official fix though....

Cheers

How did you manage to use the hack with Envshade? Does not seem to work for me, i get a black screen. Very frustrating to have to mess with files to fix something that was literally just fixed.
hotbso
Posts: 28
Joined: Tue May 14, 2019 9:01 am

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by hotbso »

I use Envshade so the the screenshot is from a file with Envshade. Did you clear your shaders folders?
disco79stu
Posts: 57
Joined: Thu Jul 16, 2020 9:05 pm

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by disco79stu »

It‘s somehow remarkable that we‘re providing eachother support for user made fixed for issues with P3D in their own forum :)
User avatar
downscc
Posts: 1623
Joined: Mon Dec 01, 2014 5:46 pm
Location: KCRP

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by downscc »

Thanks for the markup.... be careful. I used the select for copy:
// #if !defined(SHD_ENHANCED_ATMOSPHERICS_BLEND) && !defined(SHD_TO_FAR_CLIP) && !defined(SHD_SKY_STARS)
#if !defined(SHD_ENHANCED_ATMOSPHERICS_BLEND) && !defined(SHD_TO_FAR_CLIP)
float distanceFogTrans = min(transmittance, CalculateExponentialFogBlend(distanceSq, cb_View.mFogDensity));

#if defined(SHD_SKY_STARS)
distanceFogTrans *= (1 - pow(cb_mDayNightInterpolant, 2));
#endif

fogInscatter += cb_View.mFogColor.rgb * distanceFogTrans;
transmittance = saturate(transmittance - distanceFogTrans);
#endif

....and failed to realize that it is missing the last line before the closing bracket:

return float4(fogInscatter, transmittance);

If you get a black screen check for this error.
Dan Downs
KCRP
User avatar
Beau Hollis
Lockheed Martin
Posts: 2452
Joined: Wed Oct 06, 2010 3:25 pm

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by Beau Hollis »

Hello all,

Sorry about this. We added the SHADE_SKY_STARS check to fix this issue with the stars, but it looks like that flag is being set true on accident by other materials. The best workaround for this would be to create new #define called SHD_SKY in ShadersHLSL\Sky.fx at the top of the file like this:
//---------------------------------------------------------------------------
// Prepar3d - Shader Effect Files
// Copyright (c) 2014, Lockheed Martin Corporation
//---------------------------------------------------------------------------
#define SHD_SKY
#include "ShaderMacros.h"
Then change line 504 in ShadersHLSL\FuncLibrary.fxh to use SHD_SKY instead of SHD_SKY_STARS like this:
#if !defined(SHD_ENHANCED_ATMOSPHERICS_BLEND) && !defined(SHD_TO_FAR_CLIP) && !defined(SHD_SKY)
Thanks,
Beau Hollis
Prepar3D Software Architect
Wise87
Posts: 19
Joined: Thu Dec 15, 2016 5:06 am

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by Wise87 »

Beau Hollis wrote: Sun Jan 23, 2022 6:18 pm Hello all,

Sorry about this. We added the SHADE_SKY_STARS check to fix this issue with the stars, but it looks like that flag is being set true on accident by other materials. The best workaround for this would be to create new #define called SHD_SKY in ShadersHLSL\Sky.fx at the top of the file like this:
//---------------------------------------------------------------------------
// Prepar3d - Shader Effect Files
// Copyright (c) 2014, Lockheed Martin Corporation
//---------------------------------------------------------------------------
#define SHD_SKY
#include "ShaderMacros.h"
Then change line 504 in ShadersHLSL\FuncLibrary.fxh to use SHD_SKY instead of SHD_SKY_STARS like this:
#if !defined(SHD_ENHANCED_ATMOSPHERICS_BLEND) && !defined(SHD_TO_FAR_CLIP) && !defined(SHD_SKY)
Thanks,
For someone who is not edit savvy does this look correct now? I also made the entry in the Sky.fx.
Image
disco79stu
Posts: 57
Joined: Thu Jul 16, 2020 9:05 pm

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by disco79stu »

Beau Hollis wrote: Sun Jan 23, 2022 6:18 pm Hello all,

Sorry about this. We added the SHADE_SKY_STARS check to fix this issue with the stars, but it looks like that flag is being set true on accident by other materials. The best workaround for this would be to create new #define called SHD_SKY in ShadersHLSL\Sky.fx at the top of the file like this:
//---------------------------------------------------------------------------
// Prepar3d - Shader Effect Files
// Copyright (c) 2014, Lockheed Martin Corporation
//---------------------------------------------------------------------------
#define SHD_SKY
#include "ShaderMacros.h"
Then change line 504 in ShadersHLSL\FuncLibrary.fxh to use SHD_SKY instead of SHD_SKY_STARS like this:
#if !defined(SHD_ENHANCED_ATMOSPHERICS_BLEND) && !defined(SHD_TO_FAR_CLIP) && !defined(SHD_SKY)
Thanks,
Working for me, thanks!
User avatar
Beau Hollis
Lockheed Martin
Posts: 2452
Joined: Wed Oct 06, 2010 3:25 pm

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by Beau Hollis »

Wise87,
Your change does look correct. Here is that whole block copy/pated from my working copy:

Code: Select all

    }
#if !defined(SHD_ENHANCED_ATMOSPHERICS_BLEND) && !defined(SHD_TO_FAR_CLIP) && !defined(SHD_SKY)
    const float distanceFogTrans = min(transmittance, CalculateExponentialFogBlend(distanceSq, cb_View.mFogDensity));
    fogInscatter += cb_View.mFogColor.rgb * distanceFogTrans;
    transmittance = saturate(transmittance - distanceFogTrans);
#endif
    return float4(fogInscatter, transmittance);
}
I didn't re-post the full set of instructions this time, but this also requires that you clear our your shader cache folder in local appdata because the shader cache doesn't check the timestamps of headers (fxh files) and won't know which fx files need to be recompiled.
Beau Hollis
Prepar3D Software Architect
User avatar
Limbu
Posts: 7
Joined: Sat Jan 01, 2022 8:23 pm

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by Limbu »

Beau Hollis wrote: Sun Jan 23, 2022 6:18 pm Hello all,
Hello! I would like to touch upon the general view of the starry sky. In Prepar v5.3 the shader rendering of the starry sky has moved from the former shader Stars.fx to the new shader Sky.fx. And at the same time unfortunately the view of the starry sky became even worse than in Prepar v4.5, while becoming similar to Prepar v3. The thing is that firstly, the new shader Sky.fx removed the function sizeScalar, and now all the stars of different brightnesses look dimensionless equal points consisting of one pixel. In this case, the starry sky appears to be composed of the main mass of stars with slightly different magnitudes of brilliance. In the Stars.fx shader, the sizeScalar variable was calculated using the StarSizeScale table based on the index variable [scaleIndex], which is not present in the new Sky.fx shader. However, this is not the main problem, because I managed to overcome this drawback myself by replacing the StarSizeScale table with my own function which calculates sizeScalar based on diffuse and thus returning the starry sky look almost like in Stars.fx. A more important problem is the recognizability of the starry sky. Has anyone ever wondered if it is possible to recognize a single constellation in the PREPAR sky? No constellation is recognizable! I am more than sure that many simmers would accept the idea of making constellations recognizable with great enthusiasm, for there are not a few fans of night flying on VFR among them. I investigated this problem last year, and came to the conclusion that the problem is caused by the table data decoder, hidden somewhere in the depths of the application itself. This decoder reads data from stars.dat file - astronomical coordinates of stars and strangely encrypted color. Here is an example of the initial fragment of the table from this file:


[Star Locations]
;Star.1234 = HR,raH,raM, raS, decD,deM,deS, mag, specType, color, name
Star.0 = 1, 00, 05, 09.9, +45, 13, 45, 6.70 , A1Vn , 0x00E8F8E8 ,
Star.1 = 2, 00, 05, 03.8, -00, 30, 11, 6.29 , gG9 , 0x15E6E6E6 ,
Star.2 = 3, 00, 05, 20.1, -05, 42, 27, 4.61 , K0IIIbCN-0.5 , 0x8EFBF5CF , 33 Psc
Star.3 = 4, 00, 05, 42.0, +13, 23, 46, 5.51 , G5III , 0x57F9F9D3 , 86 Peg
Star.4 = 5, 00, 06, 16.0, +58, 26, 12, 5.96 , G5V , 0x34F9F9D3 ,
Star.5 = 6, 00, 06, 19.0, -49, 04, 30, 5.70 , G1IV , 0x49F9F9D3 ,
Star.6 = 7, 00, 06, 26.5, +64, 11, 46, 5.59 , B9III , 0x51E8EAF8 , 10 Cas



The decoder takes the color data as hex code from the "color" column on the right, converts it to a diffuse value, and passes it to the shaders. I tried to systematize the data in the "color" column using exel tables, but I haven't managed to figure out the exact principle of coding, and all my attempts to edit the code in the "color" column were unsuccessful. As a result, I came to a conclusion that either the data is wrong (which is less likely) or the decoder is not working properly (which is more likely). So I have long had the idea how to fix this situation, but without LM I personally can not fix anything of course. But I could for example save some time for LM developers (if that's a problem) by doing some of the work in the shaders. And the idea of correction is very simple - no need to bother with this outdated decoder, it should be discarded and just pass to the shaders decimal value of astronomical brightness of stars, which is located in the "mag" column! This value can be passed to the shaders as an intermediate variable, calling it whatever you like. Then, for example, I shouldn't have much trouble coming up with a function that converts the value of "mag" and passes it to diffuse. As for the color of stars, believe me - in reality, the human eye can barely distinguish the color of the brightest 30-50 stars, so to give color to the entire mass of stars makes no sense. And this is also one of the main reasons why I called the mentioned decoder obsolete and wrong. But it is possible of course to give color to the brightest stars, for example in the amount of 50. To do this you need to pass to shaders and the value of column "specType", but it's better to create an additional column in stars.dat, where the color data of the brightest 50 stars will be presented in an understandable way for the developer. Color data can also be represented as encrypted indexes in stars.dat, and in shaders you can make a table of decoding these indexes into float3 color filter values. As you see, there's plenty of room for imagination, and the routine and creative work can be done by outside enthusiasts, while all you need to do is to pass data of "mag" column and data of one more additional column for color into shaders. And that's only a very small part of the work that third-party enthusiasts can do to improve the look of the Prepar world. And I hope that LM is as interested in making the Prepar world better than MFS2020 as third-party Prepar enthusiasts are, so I encourage you to cooperate.
User avatar
Beau Hollis
Lockheed Martin
Posts: 2452
Joined: Wed Oct 06, 2010 3:25 pm

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by Beau Hollis »

Limbu,

Thanks for the feedback. If no color is provided in the dat file, we calculate a gray color based on global intensity and alpha value is replaced by magnitude scaled and clamped so that values linearly scale from 0 at mag 8.0 to 255 at mag 2.0. From the entries you posted it looks like there may be a similar relationship with alpha and magnitude at least for some specType groups.

In the future we may add the magnitude data into the vertex and update the shader to calculate the correct luminance value and/or size. I've made a ticket to look into this.

For now though, if you want force our shader to get valid mag values, you try setting all the color values to 0. All stars would be the same color, but you would get a consistent mag value passed into the alpha.

Thanks
Beau Hollis
Prepar3D Software Architect
User avatar
Limbu
Posts: 7
Joined: Sat Jan 01, 2022 8:23 pm

Re: P3D v5.3 HF2 | No haze/atmo (again)

Post by Limbu »

[quote="Beau Hollis" post_id=253457 time=1643893661 user_id=124]
Limbu,

Thanks for the feedback.
[/quote]
Greetings ! First of all I want to thank you for the idea, and to report that I finally finished the work on improving the starry sky Prepar. The complex approach to the solution of the problem has not bad results. Files stars.dat and Sky.fx were edited. And both files can be said to have been edited extensively. When I read your post, the first thing I had questions about was the scaling of the mag value in the application engine. You specified a scaling range from +8 at an output value of 0 to +2 at an output value of 255. First of all I had a question - are the values of the brighter stars limited to 255 ? I wanted to ask this question, but since I'm used to working this out for myself I figured it out for myself, even if it meant wasting my time. But the question was intriguing and interesting, because I intuitively suspected a scaling problem. Before pointing out the problem, I want to remind you what mag is: mag is stellar magnitude, a dimensionless logarithmic quantity used in astronomy to estimate the brightness of stars. The smaller the value of the stellar magnitude, the brighter the star in question. The faintest stars that the naked human eye can discern have a magnitude of about +6.0. At a linear scale of +8, we see too many faint stars that the naked eye should not see. But this is not the main problem, as it can be solved by repeated non-linear scaling with a shift in the shaders. Generally speaking, the nonlinear transformation must be done in any case, because mag is a logarithmic value, because a change in brightness by the same number of times is perceived by the eye as a change by the same amount (Weber-Fechner's law). At the other end of the range, limiting mag to +2.0 will cause stars brighter than +2.0 to be visually indistinguishable from one another in the sky. Although there are relatively few stars in the range brighter than +2.0, since they are the most visible stars, they constitute the main picture of the sky and should be displayed as accurately as possible. (Some of the brightest stars in the sky such as Sirius and Canopus even have negative magnitude). Therefore, after some indirect research, I came to the conclusion that the value of mag is still limited at +2.0. I will not describe now the details of my experiments, but they clearly indicated the presence of such a problem. That's why I came to a conclusion about the necessity of editing the value of mag also in the table stars.dat. For a start, I made a version of stars.dat with the number of stars 1809, in which I excluded stars weaker than +5 with the help of spreadsheets. In this version I simply performed an arithmetic linear shift of Mag by two magnitudes upwards. (I added a value of 2.001) Although this version was more or less satisfactory to me at first, but then I thought that a version with a more complete number of stars has a right to exist. For example, in the case of a dark moonless night and clear atmosphere I want to see more stars. But in the case of the original number of stars.dat(9096), a simple arithmetic shift shifts a significant number of stars into the invisible area. This creates two contradictory conditions - at one end of the range in the area of the dimmest stars the value of mag should be left unchanged, and in the area of the brightest stars a correction of about +2 is needed. To satisfy both conditions I invented a non-linear correction of the mag value by a formula I invented myself, and edited the mag values with the same spreadsheets like exel. Then I wrote and finalized corresponding code in Sky.fx shader. I also added that in the table stars.dat I additionally edited the mag values for ten of the brightest stars in the sky, so that the view of the starry sky was as close as possible to the real one. I corrected also coordinates of Orion's belt stars, which for unknown reasons looked distorted (although their coordinates seem to correspond to those from other astronomical handbooks). As I am well acquainted with astronomy from my childhood, I know all constellations and bright stars except the southernmost ones very well, therefore I consider such visual correction quite appropriate. Perhaps something else should be tweaked, but no time now.
In the Sky.fx shader you can adjust the overall brightness of stars and other parameters of their display. In addition, brightness and other parameters can be additionally tied to some weather conditions and the location of the observer, but this is not yet implemented.

Since I do not have the opportunity to publish screenshots on your forum in a convenient way for me, the screenshots related to the results of my work, I published in my thread. [url]https://www.avsim.su/forum/topic/189159 ... omLogout=1 [/url]

If you are interested in my work, I can give you the appropriate files: Two versions of edited stars.dat, and edited shader Sky.fx. By the way, I added my other tweaks to the Sky.fx shader - a tweak for smooth and correct ignition of stars, and a tweak to reduce brightness of stars near the horizon. So in a private correspondence I can add about other technical details of my work, but as I am a newcomer to the forum and do not speak English, it is not clear to me yet - whether I have the opportunity to write to you in contact and transfer files. So I have a request to give me clarification on how to contact you.

And once again about the correction made in the table stars.dat. If you make correct scaling of mag value in the application code, then of course editing of mag values like I did in stars.dat table is not needed. But nevertheless I still recommend to entrust such work to enthusiasts, and since they can't edit code in application, it follows from this that I ask you to pass mag value from stars.dat table to shaders in unchanged form. At least as an additional option. And of course we can think about passing to shaders information about color from additional column stars.dat also as a separate variable, and what to do with it later the enthusiasts also will figure it out.

Translated with www.DeepL.com/Translator (free version)
Post Reply