Shadow System Research

NMS doesn’t use random rolls for things like terrain / planet generation. Most of what you see is pre-baked in the seed and associated algorithms. Otherwise we would all be finding different systems at the same co-ordinates.

There may be some random elements in the placement of minor scenery like rocks and plants, or maybe crashed ships - but even there, I’m not convinced.

My own opinion, for what it’s worth, is that absolutely nothing in the game is random. But that the algorithms governing world generation are extremely sophisticated, and give rise to a level of complexity that looks like random chance.

It’s the only way we could all occupy the same, identical, universe.

5 Likes

Slight misunderstanding in nomenclature. When I say “roll”, I mean generating a random number. It’s a common illness among people that (try to) design games, because we derive a lot of mechanics from dice rolls (very clear probability distributions, and most of all it allows you to test mechanics on paper before committing to actually investing the time to code them).
The concept is the same, however. You generate a random number from a seeded generator. If the seed is the same, the resulting sequence is always the same. Hence the disruption when you insert a step , because it breaks your sequence, and things that are “rolled up” afterwards get different results than before. Whether you call it a “roll” or “generating a random number” doesn’t really matter.

That’s splitting hairs, really. Nothing a computer can do is truly random, that’s why the technically correct term is “pseudo-random”, but programmers are lazy and can’t be bothered to use the pseudo-qualifier all the time, since it’s kind of redundant when used in a computing context.

6 Likes

I’m not being deliberately argumentative here. For me, it’s quite fundamental to my understanding of the way the game works. Parts of it look random, but I don’t believe they are.

Regarding random number generation, there’s lots of ways that have been used in the past. Whilst they’re not truly random, they are entirely unpredictable, and so as close to random as you’ll ever need.

Probably the best method is the tried and tested key press. You require the user to press a key. At the time the key is pressed, you capture some piece of fast-moving information. Say, the time in hundredths of a second. If you want to get more random, you could also capture the position of the hard disc read heads at that moment. Add to that the IP address of the last website the user visited. Multiply them all together.

And yes, it’s not truly random. It could be reproduced. It could be faked. But for all practical purposes, it’s as random as you’ll ever have a use for.

4 Likes

Only the Shadow knows!

This has turned into a great topic by the way.

Wouldn’t a CSD Shadow base be cool!

If a Comm Ball at the base area could hold a URL, a shortened URL to this topic would be epic.

5 Likes

I am not even close to finishing up the research, because of the sheer amount of regions as well as systems. Doing proper comparisons is going to take some time. I have been intrigued by these shadow systems for a very long time, but only now really got it started with documenting data.

I will certainly let you all know what I have discovered so far, just wanted to leave a quick reply at least, as I should actually be asleep right now.

Are those values about the region coordinates for groups of system counts, and if so, what axis are they representing? Or am I misunderstanding what you are trying to say here? I have personally not really noticed any changes in regions.

I don’t believe this to be related at all, but more likely a different way of interpreting if a system is valid or not. To me a system is valid if it is shown in the Galaxy Map and can therefor be warped to. The system at 013D:0084:0675:00EC, Rihowley, is perfectly valid to me.

I have not tested if a Portal will error for it, but that is actually what got me curious about these systems in the first place. Not all planets have a portal, contrary to popular believe, even more interesting if a system doesn’t have one.

Appreciate you chiming in, and would love to compare finds, not just within a single galaxy, but across galaxies as well, as there are some interesting patterns.

More to follow soon … sleepy time now … :sleeping:

4 Likes

In my experience, trying to portal to any “out-of bounds” address does appear to throw an error and redirect to a nearby system. A basic example is 000000000000, which randomly (@Polyphemus, @jedidia :wink: ) redirects to one of many possible systems surrounding the core.

Really? Do you have any examples? Is the game just unable to locate some via the monolith method, or is it also impossible to portal to certain planets by changing the portal address’ 1st digit too? I know there used to be a bug that prevented portalling to the last-numbered planet in any system. AFAIK those planets always had functioning (outgoing) portals though, and the bug eventually got fixed.

3 Likes

I understand that. That’s why I said we had a misunderstanding in nomenclature. We are talking about the exact same thing.

No on that last sentence. Big, big, BIG no! What you suggest is way too little entropy to be considered secure enough for encryption (which happens to be a practical purpose we all make use of, even if we don’t notice it much).
But in the end, that’s just splitting the hair in the other direction, isn’t it? We’re still generating a sequence of numbers from a seed, and the sequence cannot be predicted in advance without effort. It’s all the same thing. It’s all “random”. The only difference between them is where I take my seed from.If certain standards have to be fulfilled by the entropy of the seed, we call it secure random. If the quality of the seed doesn’t really matter, but we need the result to be “unpredictable” (by any reasonable human standards), we call it inconsistently random (which is semantically misleading, I’ll admit). When we need the sequence to be consistent every time, we call it consistently random. Meanwhile, we also call all of these just “random” for short, once the context of use has been established.
Since this is a forum for a procedurally gnerated game, it didn’t even occur to me that the context of “consistently random” might not be implicitly understood by everyone, so I guess I really was just too unprecise in my wording out of habit.

5 Likes

I have used the portal to determine whether a system is valid. If I want to know how many valid systems there are in a region I increase the SSI in a portal address until the portal complains that the address is invalid. I was not aware that there was a difference between the systems seen in the GM and systems shown as valid via the portal.

I should have been more clear that I was using the portal for my experiments. I meant that Rihowley is not accessible via the portal.

5 Likes

At times I create a base without having a location for it yet. Once I finish my base, I would then search for a specific location to fit this base. Back in the days, it was very useful to have a Portal near a base, for others to easily visit. This has always been reason for me to check the portal location, once I found a suitable planet. I then noticed how a planet would not always actually have a portal, and on rare occasion even the system would fail.

I should have some examples of these somewhere, written down in one of my many notes, although we now have a great example with Rihowley (3 planets) mentioned earlier, I also noticed how Xaine at some point discovered this same awkwardness of a planet erroring when trying to portal to it. He even turned it into a community system.

I don’t think there is supposed to be a difference, but it comes to show that not all planets or systems result in having a portal available for whatever reason, whether intended or not :crazy_face:

4 Likes

Finally found some time to start listing some of my findings so far.

  • Black Hole system will always have SSI 079 (hex) = 121 (dec).
    • It appears there is only ever one Black Hole system per region.
  • Atlas Interface system will always have SSI 07A (hex) = 122 (dec).
    • It appears there is only ever one Atlas Interface system per region.
  • All systems with SSI 000 - 07A (hex) = 0 - 122 (dec) will always be yellow.
  • All systems with SSI 07B (hex) = 123 (dec) and higher will have a chance to be coloured.
    Not sure what the exact odds for the colours are, but from my main research system, about 70% is yellow and 30% is coloured (Red, Green, or Blue).
    For Abandoned and Uncharted the probabilities are as follows:
Colour Abandoned Uncharted
Yellow 0 0
Red 0 0.95
Green 0.1 0.4
Blue 0.1 0.4
  • Shadow systems are systems you would normally not be able to get to. This is only possible by editing the save to have the player spawn there.
  • Shadow systems are not visible in the Galaxy Map.
  • Shadow systems do have a portal glyph sequence, but when used with a Portal, will always error and instead send you to a random ‘valid’ system in that region.
  • If the player is in a shadow system and then uses a portal, returning through the portal will get them back to the shadow system.
  • If a base is created in a shadow system, teleport will work fine.
  • The SSI where shadow systems start, can vary per region. In my main research system, it starts at SSI 0ED (hex) = 237 (dec), but in the exact same region in Hilbert, it starts at 226 (hex) = 550 (dec).
  • The max SSI value for (shadow) systems in a region is FFF (hex) = 4095 (dec).
    • Any value higher will always point to SSI 4095, even though the value will remain unchanged in the save file.
    • Any negative value is internally subtracted from 4095 (dec) and can therefor point to different shadow systems and even to valid systems. -4094 will for example point to SSI 001, the first valid system in a region. These negative values also remain unchanged in the save file.
  • Setting a waypoint to a shadow system in the Discovery tab, will always point to the first system in the region.
  • The Space Station suffix name is always the same for any system within that region, if it has one.
  • The Space Station suffix name will even be the same in another galaxy for that exact same region. The Space Station model used will be different though.
  • The Spectral Class for the respective coloured systems, will be ‘set’ for the full region. So if a region’s yellow system uses F as Spectral Class, all yellow systems in that region will then use F. If a red system in that region uses M as Spectral Class, all red systems in that region will use M.
  • The Spectral Class for the exact same region in a different galaxy, will not necessarily be the same and can be different but will still be ‘set’.
  • Besides the Space Station naming above, it appears that all regions and the systems within are being procedurally generated across galaxies, and therefor differ for any other specifics. Of course the same rules still apply.

I may have forgotten some details, but for now the above are my main findings from the research done.

There is still more I wish to investigate, which requires comparison with other regions. I do for example wish to check core (even those hidden from sight, known as the ‘haze’) or near core regions, but also those at the outer edge of a galaxy, known as the ‘fade’.

9 Likes

This is fantastic stuff.

In my fic, I’m going to have Nigel encounter a character doing these kinds of research. May I use your findings in the story? With credit of course.

And a question. Are the Abandoned and/or Uncharted systems and Stations populated?

3 Likes

Of course you can use my findings in you story, you do not have to credit me either.

Abandoned systems used to be inhabited but all NPCs have left. The Space Station is damaged and has red interior lighting. There is still a Trade Terminal and a Teleporter present, nut there is no market. Trading Posts and Transmission Towers can still be found and will be functional, although most likely now infested. Buried tech can also still be found. Pirates are also present.
Source Wiki
Note: So far it appears there will only ever be 2 Abandoned systems in a region, although I need more data to be able to fully confirm.

Uncharted systems are worse and devoid of any life, apart from sentinels. There will not be a Space Station at all and many planetary points of interest are absent. There is no buried tech either.
Source Wiki

4 Likes

Question about that:

What happen if in a shadow system and opening the galaxymap ? Normally you can’t see yourself at start point. Does it crash the game ?

And can you claim a shadow system when discovered ? Can you enter a shadow system in https://galacticatlas.nomanssky.com ?

3 Likes

If you open the Galaxy Map when in a shadow system, it will always default to a specific system for that region or close to that region, which is not the same system it would show when setting a waypoint to it. The system shown however, does not show you as being there, even though the ‘player icon’ is shown with it. It just shows the system in the middle of the screen, ready to be clicked/selected. So the Galaxy Map does work, you are just not in any of those systems shown.

In my initial research region (Untheodos Boundary in Euclid) it would always show Miawanum from a neighbouring region.

I can upload the discovery of those systems, even rename them, as you’d expect. I have not uploaded them all though, nor renamed them, as it would complicate research. I have not tested any other discoveries like Fauna, Flora, or Minerals, but I do expect this to work just fine.

Sure, can enter the portal address just fine and appears to give the correct location as well. No restrictions are built-in like some apps do like NMS Portal Decoder for addresses with SSI > 2FF (hex) = 767 (dec). The wiki mentions the same restriction of 2FF for SSI and I am curious why, as I have not seen this restriction. It could possibly be the max SSI value for ‘valid’ systems, which would require me to do more research to at least get an indication of this being true.

4 Likes

Here are some images from the same region, taken in several different Galaxies. You can see how the icons and pathing lines all remain visible (although limited in amount, as there should be way more). You can also see how distribution and location of the systems is different.

Euclid

Hilbert

Calypso

Hesperius

Hyades

PS: The icons of ‘visited’ systems as well as the pathing lines remaining visible, is a bug. For some reason these do not get refreshed/updated.

4 Likes

This is super cool, you’re slowly reverse engineering the internal procgen rules! :exploding_head:

5 Likes

The Vorlons are extremely interested in these Shadow systems. They want to restore light to darkness, order to chaos.

Their primary star system:
OshKosh

6 Likes

Oh, I can help with this, I can refute it! There are at least 7 abandoned systems in the Tadorne Void, a region near the Expedition 1 Rerun start. There are 3 abandoned systems clustered right around Cutorpet (0948:0086:0ED2:00E5), and I’ve seen another 4 scattered about higher up. All green or blue stars.

4 Likes

Thanks for that info, will investigate, with 7 being quite extraordinary to be honest!

Edit:

Yeah, that is the defined game rule, yellow and red systems are not supposed to be abandoned :wink:

3 Likes

Thanks again @Argh , I have so far indeed found 7 Abandoned systems in the region you mentioned. This is great and quite a large amount. I do have a feeling this is a rather large region with so far at least 347 valid systems. Will require some more investigating!
You can check the sheet linked earlier

5 Likes