Data visualisation

Project Skyscraper
1

I was looking at @vector_cmdr 's cool site https://project-skyscraper.vectorcmdr.xyz/
The “View: neural_net” section shows a graph, how did you generate that?

IMG_9553
IMG_95531170×2075 201 KB

IMG_9552
IMG_95521170×2532 483 KB

Help me understand what I am looking at: Each node is a label (such as system… or no_category or home) plus one post? And the edges connect post nodes to a common label node (dataset_2016)? I see you already use colours for media/post/page, that’s good.

Obviously (this always happens) the first generated graph is always a mess of lines and hard to scan. :see_no_evil_monkey:

So I wanted to start this thread to find out a) if you automated this and b) if we can design an automated, better readable, ARG-forever-clarifying graph together? :joy: Separate pages of graphs surfacing a variety of underlying patterns.

Examples for what I mean:

Time patterns:

  • One timeline of posts in the order they were revealed to us in real time
  • One timeline of posts in the order of their datestamps
  • extra dimension: authors distinguished by colour
  • Next to each other, so we could compare the timelines?

Label patterns

  • label nodes are the centres
  • Edges point at posts baring that label
  • This is possibly what you currently have? It maps one-to-many (each post can have none or several labels) so high chance of looking messy.
  • Here I would like to brainstorm with you how to add a dimension to surface something useful

Author patterns:

  • author name nodes are centres
  • edges point to their posts
  • extra dimension: Post nodes colourised by label? or date (gradient)…?
  • Since each post has only one author in WP, that would result in 2-3 separate starshaped clusters
  • We could see if one author posts a lot and another rarely, or if one posts about one topic and not the other, etc.

See what I mean? Assuming vectorcmdr’s tool allows for this, what could we use it for, ideas? :slight_smile:

5 Likes
2

Eventually you will reach an event horizon of complexity, and disappear up your own black hole.

Not necessarily informative, but certainly different.

9 Likes
3

It’s generated from the sitemap + post/page links/backlinks with some filters and such applied and populated into the node pool (from my monitor/mirror daemon).

From there, each “connection” is drawn between each node that it shares a connection with, which is why it’s a spaghetti mess now with the eleventy-seven 2016_dataset posts/pages on initial load.

Under the content name is the content slug, and hovering over a node displays a tooltip card with extra info.

Clicking a node isolates that node and its direct connections (redrawing a mini graph). It also displays a fixed card with link and connection info in the top right.

Esc or clicking off returns to the view, but a search filter (bottom) is always active if there is a string entered in there.

The daemon occasionally regenerates the graph as things are logged and changed and pushes them to GitHub where pages rebuilds and deploys the served site.

You can do most of the above by clicking to focus their nodes, such as search “architect” and click the WP author name node, which will reproduce the focused graph with only their associated posts, etc.

The post ID rundowns, mirrored file manifest, etc. are in markdown reports in the repo itself here (I should add some links for these) for viewing the posts by ID, timestamp, initial, modified, author, series cadence, etc.

I could add more specific sub-graphs to it as time permits if it’s something people feel they would use.

EDIT: It’s worth mentioning, while it works on mobile, it’s much less data dense / more usable on desktop (which it’s really designed for).

5 Likes
4

Awesome work, thanks! Yeah I was on mobile. But at least it kinda works on mobile, it’s just a suboptimal small screen.

We’re getting so many new posts, this might just save our brains from exploding… :face_with_hand_over_mouth:

Can you possibly make a timeline or calendar, so we can track which posts are full and which empty? In another thread, someone posted this calendar-like grid with filled-in boxes–or whatever you can get to keep track of all these dates?

2 Likes
5

Yes yes sure, thing is, I love spaghettification, and Italian food is great during space travel, and on the way down into the black hole, I will have infinite time to sort the data by ALL the patterns, you see? :star_struck: :zany_face:

5 Likes
6

I forgot to reply to this part before (easily distracted sometimes!), but for the un/informative nature of it, I think it probably helps to think of it as a spreadsheet in visual form (much like any graph) for anyone that sees it and goes “what is this mess?”.

Colour → column
Node → row + cell combo
Selection → filter
Search → search
etc.

It’s definitely a for-purpose kind of tool that doesn’t have a broader application beyond it’s singular purpose. More of a scalpel than a leatherman.

For the sorts of queries and patterns we’ve been looking more deeply at on NMScord, it’s fast and useful for seeing associated items and patterns without sifting through a sheet, or seeing outlier media types, external sites, etc. at a glance instead of loading up multiple pages and tabs in-browser - especially so for the clusters of datasets.

For instance; looking at unverified / stable is a click and you have all your data right there instead of looking to a ‘status’ column and surfing it with ctrl+f.

But yeah, certainly not everyone’s cut of tea (or mug of muffin :grin:).

8 Likes
7

Is that made using Obsidian? This looks like my D&D campaign planning :rofl:

4 Likes
8

I know that feeling well lol. The campaign notebook can get pretty crazy.

It’s built using D3js - it’s great for all sorts of dataviz.

6 Likes