WikiGraph Browser

Method for visualizing the connections between pages in a WikiSpace, and ideally connections to other wiki spaces (Sister Sites).

Outcome - update: I turned this off because I ran into issues with case-sensitivity at some point and didn't want to fix the code - but you can still start from a fixed page at http://www.wikigraph.net/wikigraph

there's a link "Click here for WikiGraphBrowser" at the end of the Backlinks box near the bottom of every page
that launches a D3.js view of the pages linked to/from that starting page
you can mouse-over any node to see its name
you can click on any node and its peers will be added to the graph, with a force-directed view updated.
(it has a glitch where the old lines don't get erased)

Goal: Wasy to browse graph of pages in a wiki space. Ideally in-browser so can easily see both the Graph Drawing and actual individual Wiki page contents.

Semi-related: WikiGraphAsDeliciousData

2013: Playing with my WikiGraph data set

Tl Dr - either

click click here for WikiGraphBrowser at the of the Backlinks box near the bottom of any page on this WikiLog to open a new browser window that shows the nodes "surrounding" that page, or
just go straight to http://www.wikigraph.net/wikigraph to see nodes surrounding My Recent Key Pages.
in either case, clicking on a node will adds its neighbors to the graph

(See WikiGraph page.)

Too many Graph Drawing options to choose from.

picking D3Js for now, as it includes data access bits, etc.
- though might be amusing to try rendering for my entire WikiLog dataset at once....
  - consider 3 "groups" of pages - My Recent Key Pages, BlogBits, and other
starting with this intro - hmm, maybe not, as that also requires Coffee Script and don't want to add extra pieces

Going to pick the links around 4 anchor pages in my WikiLog

want to update my list of page names (and catch some key pages which seem to be missing so far)
- update - nah don't bother, not necessary at this stage
do a query, then write output to a static JSON file for now
- http://pymotw.com/2/json/
- http://docs.python.org/2/library/json.html
- actually, will probably just hack more directly in Python to copy exactly the example I'm working with. Raw file generated Apr07'2013.

Re-evaluating d3 example choices - not even sure what look I want

basic network visualization
want easily viewed node labels
auto-layout - force-directed
probably no groupings of nodes that drives placement (though up above I consider 3 groups which I might use to assign color)
rather like this visualization by cgmartin of ITunes artist/genre data, so will play with that.

Apr07: playing with cgmartin's example

download from GitHub, stick down inside /static/ directory of my local WebPy area
try to just open index.html in browser without Web Server. Get no rendering.
use WebPy to deliver page - now get rendering.
start dropping attributes that aren't applicable
- list of genres within each artist-node
- drop count - drives songsTypeSize(), so just have that function always return 0.5 for now (and remove from rendering of node label)
- drop plays - drives playsTypeSize(), so just have always return 0.5
- have radius() just return 10 for now
- drop rating - does that matter?
change node types to k (key), b (blog), w (wiki/other) - change songsTypeSize() and playsTypeSize() and radius() to have different set values based on type.
write code to generate JSON from my WikiGraph subset (211 links, makes 170 distinct nodes)
- just writing str() of a dictionary, not using JSON library
  - just using pagename as both name and id for each node, rather than giving integer value, and so of course link sources and targets are pagenames.
  - after-the-fact, change ' to " by grep
- fail! cannot call method 'push' of undefined - from line neighbors[o.source.index].push(o.target);
- change my Python code to generate JSON that has an integer id for each node - now it works
  - but this will have a downstream problem, once we get to the point of dynamically bringing in more nodes.
also note have some sort of scaling issue - can't get all the nodes to fit in the window

Apr09: Next step: what I should have done in the first place: fork his system on GitHub, make my specific version, share for input and help.

Stick my past snapshot in side-directory.
Fork project. Renamed to https://github.com/Bill Seitz/WikiGraphBrowser
Made some tweaks right in the GitHub web UI (change some file names)
Do the clone, add upstream, fetch bits associated with forking.
It works, using that code/data.
Do some diffs from my snapshot into the repository.
Now see nothing in the JavaScript window. Console says Uncaught Syntax Error: Unexpected token ' It seems to be associating this with index.html not even the JavaScript file?
- No ' in the index.html
- See some comments about invisible characters in the source. Convert index.html to ASCII, save as UTF8 with Unix line-endings. No diff.
- Found a missing ' in the JavaScript file! Fixed that, still didn't change result!
- Saw some people talking about problems with tabs in source code, just when using Chrome?!? Detabbed. No change.
- Boy, shocking how useless Chrome's dev tools are for this!
- FireFox gives a clue(?) around callback(text ? JSON.parse(text) : null);....
- Ah, the issue was that I ended up with a JSON graph file where I hadn't converted the ' to ". Now works!
Apr16: committed, upstreamed to GitHub. Loaded online for viewing.

Apr16: regenerating JSON file with text node IDs.

hmm, reading first - not sure this is supposed to work. First sample I'd looked at didn't assign numeric IDs to nodes, but the links still used numerics!
docs for force.start() say The layout also initializes the source and target attributes on the associated links: for convenience, these attributes may be specified as a numeric index rather than a direct link, such that the nodes and links can be read-in from a JSON file or other static description that may not allow circular linking. The source and target attributes on incoming links are only replaced with the corresponding entries in nodes if these attributes are numbers; thus, these attributes on existing links are unaffected when the layout is restarted.
Perhaps some pre-processor code in the JavaScript to take JSON with labels and turn them into the integers?
- Or to actually attach (copy?) the nodes to the links? This may provide some inspiration... Except that other bits of the cgmartin code seem to assume the numeric structure... though if my code runs before that it may be ok. Also will look at cgmartin state just before it builds linkIndexes[]....
- The attach/copy of nodes to links worked! (At least at this stage.) Committed/pushed.

Apr17: working on doing query with WebPy and returning data to D3. (This part isn't in GitHub)

have WebPy taking pagename as part of URL call and generating JSON, ready to pass to D3.
hit some snags, and as thinking about it, realize might want different architecture (thinner container page, separate URL/service to deliver the data), for sake of adding new nodes to the page.
also need to consider that initial click... maybe just put generic link in wiki page, then WebPy uses Referer to grab the launch page, passing just the pagename to JavaScript, which then calls the JSON service....
(also, for that initial load, thinking about doing 2 generations of query right up front - grab the links around the trigger page, plus the links around each of those linked-nodes... (for one thing, that would expose links across the 1st gen siblings))
- midway-idea - do that 2nd query, but only for links among 1st gen siblings, without adding new nodes
  - on the other hand, that makes clicking on a node state-ful because it has to seek links only to pre-existing nodes. So may start by just doing 2 full level of search, just to see what happens.
so hg commit what I have and fall back to some research.
next step: re-architect to have the JSON-returning script with initial pagename passed.
Working! Referer drives 1-name query. (Only doing 1-gen for now.) Committed.
next step - research mouse events (already highlights based on click), add events for query-on-clicked-node and open-clicked-node-page.

Mouse actions (Apr27)

current code {{{ .on('mouseover', fade(true)) .on('mouseout', fade(false)) }}}
Another example: .on("click", function(d,i) { alert("Hello world"); })
Another example, focusing on select(this) vs select(d) for acting on data vs layout item.
hacked some bits together - it works, except that it's just making the new cluster the whole thing and leaving the old cluster behind. Have to add the new nodes/links, derp...
then just tried "adding" root = root + json then root.nodes = root.nodes + json.nodes but it looks like adding is the wrong operator (Apr28)
then tried looping through json.nodes (and json.links) with root.nodes.push(node);
- hmm, seems like the links from the first pass through have lost all their content (source and target are undefined), while the new links are fine
  - pretty sure this is related to my code that copies nodes into links (d.source = node_hash[d.source];) - need to rebuild from scratch, or save state across clicks, or something....
- had to make my dictionary of node ids global, derp
- then had to handle dupes (Apr30)
- now looks/words pretty good! Except...
  - previous generation of links are still drawn on-screen
  - overall graph doesn't end up in good location often - big chunks slide offscreen even though they didn't need to.
  - (also still have to deal with idea of ctl-click on node to go to its page, plus want to keep more node labels always-visible....)

May02: took code-as-is and made it live. Added link to end of BackLinks JavaScript.

Jan'2014 - want to change behavior of node labels

Current behavior

when first launch (1 node in center, links surrounding): no labels
- mouseover center: all surrounding labels turn on, then couple seconds later the node itself gets a (differently styled) label
- mouseover non-center node: center gets label immediately, then couple seconds later moused-over node gets differently-styled label
click on a node to add its cluster - same behavior
to generalize: when you mouseover a node, all the nodes connected to it have their labels appear, then a couple seconds later the moused-over-node gets its special label

Desired:

when first launch, show label for center node
when mouseover a node, show its label in same style/timing as connected nodes
when click to add a cluster, that cluster-center gets label that never turns off, and all previous cluster-centers still have their labels, too

2003: Touch Graph Wiki Graph Browser

old inspiration: HotSauce

Edited: 2022-06-21 20:21:36.736772 | Tweet this! | Search Twitter for discussion

Bill Seitz