WebSeitz/wiki

TwitterBot

Daemon that auto-posts to Twitter.

Hilary Mason Python code: https://github.com/hmason/botomatic

Leonard Richardson Sycorax - http://www.crummy.com/software/sycorax/

Buster Benson - 2013-09-26-BensonMarkovChainTweets

Mar'2013 Darius Kazemi: TwitterBot etiquette

May'2014 Mark Sample: A protest bot is a bot so specific you can’t mistake it for BullShit

Apr'2014 experimenting

Crude goal: make TwitterBot to promote Private Wiki Notebook by periodically tweeting quotes from the content. Use Leonard Richardson's olipy Python code.

Fork Leonard's code. Install python-twitter library. Install Text Blob library.

Looking at his example.ebooks.py code, will probably want to create a single text file with all the Private Wiki Notebook contents pulled in. Strip links, etc.

Try to run example.ebooks.py, get error

 File "/Users/billseitz/Documents/djcode/st/olipy/tokenizer.py", line 3, in <module>
 from text.base import Base Tokenizer
 Import Error: No module named base

Hrm, if I try just import text it works fine.

Change tokenizer.py to have from textblob.base import Base Tokenizer - now it runs. Found 65 quotes.

Next - generate Private Wiki Notebook content file.

  • decide to use local cache of HTML pages.
  • write little script to combine them all into 1 file
  • start doing regexp in editor to strip tags

Now try using his code.

Grr my text isn't anything like the Project Gutenberg text, and if I try to just drop the Gutenberg layer then I don't have any structures defined...

  • once I removed the text-name param I was passing, it ran
  • tweaking my text
    • I use lots of phrase-only lines - I put a period at the end of each now
    • remove all URLs

Hmm, perhaps a bit too surreal?

Let's go back to his example output:

Tite Street, Chelsea, but the
settled  home, until in 1902 he once again took a house in
house with him, for his health was  breaking, and he was in need of companionship
had a  house that is no longer standing at 47 Queen Anne
Autobiography how he came to Chelsea, and gives a  glowing description of his house
liveliest dialogue on
terminate the moment you are bound to go. A most
occasion a man came to mow
Georges, The Virginians
letters tell of how  he is house-hunting, and in the intervals working "at racehorse speed" on  Barnaby
more public movements; acting
year after. Gore House has vanished from its place  long since, and the Albert Hall more than covers the site of
best good  man with the worst- natured Muse,' being 'The worst good man with the  best- natured Muse.'
Hereford Square,  South

And here's some of mine:

A Wiki For Your Notebook
section of Hack Your Life With A Private Wiki Notebook Getting Things Done And Other
Automatic Linking!.  Wiki typically does this through the Smashed Together Words approach. You Link As You Think. Then you, or someone
Opposing Anti Patten: Free Link.
capitalize each component word), if your Wiki Engine supports Automatic Linking.
But there are plenty of edge-cases
requiring examination - see Wiki Name Examples.  Job Creation.  Historically, driven by SmallCo formation and growth.  Vs Jobless
Covey’s First Things First. (Some tactics for handling your change in direction/life.).  Wiki Text. Smart Ascii used for Wiki
for much larger chunks of text (Wiki, WebLog) than was
Authoring within Content Management System
Text Rules.  Re Structured Text.  SeText.  Wiki Wiki Web:Te xtFormattingRules.  Mark Down. Textile (orig and Mark Pilgrim's Python clone ).
Pilgrim's Python clone ).
see Meatball Wiki:WikiMarkupStandard.

Try the Markov Chain code (Leonard says: Queneau assembly is usually better than a Markov chain above the word level (constructing paragraphs from sentences) and below the word level (constructing words from phonemes), but Markov chains are usually better when assembling sequences of words.), get:

It is like a designer. Any thing behind these signature strengths to describe a big HTML v4, XHTML, XML, PDF, etc.
 And I'm hoping to design of those industry structures changes for each phrase (phrase that using Wiki Page for a Wiki Have a Synergy phase where wiki-newbies feel as soon as a hack. It was written in Ideas" issue that implements a blog. It all these?
Became PbWorks.
Usable Keyboard, if you can help more later using Camel Case model of other high-level pages in deciding which concludes with it or Context of ...") ("hmm, why I find any WebApp (e.g. Team Wiki node at our economic/Un Employment issues covered on my laptop, and has a way for lots of Our Marketing Plan, etc. (If it's best possible each great surge .
What doesn't create your Cell Phone becomes a service, you'll sometimes educational institutions. That last row of good person might change.
Zen And Other Systems:
Wiki tends to be notified of the United States.[2] He uses v1 (RDF) - see the header.
  “I actually "accomplished" anything else may be nice if we use the trombone. In addition, it like the creation of Self Improvement by SmallCo formation and a number of their work nicely when you just found them in a Positive Psychology is a newbie, or when having lots of Cognitive Surplus and seems of people.
Marcus doing something, I think it's Always Innovating Smart Ascii type, ability to change in constant reciprocal action with your Photos in email).
 He now and '\n' to give it doesn't have to refer to continue until you reduce the same thing behind 1 or you a place again to define success that Weekly Review section up the world economy in this was forced on his "final" 6 percent by the difference in a manually coded Wiki.
 at any Winner Take All Work Week.
 - a new staff discover a Synergy phase to relax and send EMail, and I haven't seen a work was joined by John Gruber and present; and.
 The SMC is doomed to give such phrase into a manually jump in Cheap Hosting, you think about Visual Language, and Smart Ascii used at a page from Sept'99. It might want to synch my Weekly Log page to roughly 50 percent of people to improve Shared Understanding... For Your Life?
 - Sense Making A tentative typology of frames interface, which includes notes are OK with enough to be bad loans.

(Also, for TwitterBot purposes, many of these are too long. Will review that layer of code later...)

Created Twitter account, manually pasted a couple nice Markov bits. https://twitter.com/WikiNotebook

Try Leonard's code:

Traceback (most recent call last):
  File "integration.py", line 18, in <module>
    class Twitter(twitter.Twitter):
Attribute Error: 'module' object has no attribute 'Twitter'

Getting Twitter API access

  • Leonard's code supplies TWITTER_CONSUMER_KEY and TWITTER_CONSUMER_SECRET, but can't see how to get my user access token without setting up my own Application, so do that anyway.
  • but can't set my API access to Read+Write without assigning my mobile number to the twitter account, and I can't do that because that mobile number is already associated with my core account.
  • grr I'm not even sure what kind of authentication model I'm supposed to be pursuing.
  • ok, not "application-only authentication" as that's read-only: With Application-only authentication you don't have the context of an authenticated user and this means that any request to API for endpoints that require user context, such as posting tweets, will not work. So I do need oAuth-signed.
  • But which one?
  • PIN-based sounds appropriate: The PIN-based OAuth flow is intended for applications which cannot access or embed a web browser in order to redirect the user to the authorization endpoint. Examples of such applications would be command-line applications, embedded systems, game consoles, and certain types of mobile apps.
  • That seems to require the same kind of setup. So decide to try register my app via my main personal Twitter account which has my mobile phone. That works. Then create my access token.
  • Hrm still have fears this will only allow me to post tweets from my core/personal account.
  • The Twee Py documentation explains a little more that fits with PIN authorization. So going back to that path. Key word = verifier
    • except that stuff is buried down in the oauth library so that doesn't tell me if there's any problem with the python-twitter library handling it.

Try Leonard's sycorax software.

  • run get_access_token.py:
Traceback (most recent call last):
  File "get_access_token.py", line 3, in <module>
    from oauth.oauth import (
Import Error: No module named oauth.oauth
  • it appears this sycorax library uses a different oauth library than the olipy library

Find this bit of instructions for python-twitter that uses PIN.

  • pip install oauth
  • pip install oauth-python-twitter
  • make edits to oauth-python-twitter.py per link
  • realize the examples in that page are just little components to build on, nothing end-to-end to test
  • also realize that installing that oauth means that sycorax code might work now!

Try sycorax again

  • have to install httplib2
  • now it works!
  • able to use get_access_token.py to get PIN

Apr28: Assemble my own tweet.py from various bits of sycorax code, plus comment-docs inside python-twitter.

  • run: twitter.TwitterError: [{u'message': u'Invalid or expired token', u'code': 89}]
  • run get_access_token.py again - GET different PIN, but end up with same twitter_token and twitter_secret. Run tweet.py again, and still get the "invalid or expired token" response.
  • ah, sycorax examples of code passed the various tokens in the wrong order. Swapped them per doc-notes in python-twitter and now works! (For one manually-defined tweet.)

Next steps

  • randomly pick from various streams - ebooks, markov, maybe direct excerpt - done
  • manually pick 1 line from that stream (many of them seem too long, so need to look at that...) - done

Create cron per this and this - a bit concerned about how use of VirtualEnv relates to this.

  • set for every 5min: */5 * * * * ~/documents/djcode/st/sycorax/tweet.py
  • save at 15:46. Has been 10min since previous tweet.
  • yep VirtualEnv Import Error: No module named twitter
  • try this, save at 16:15.
    • now at */5 * * * * ~/documents/djcode/st/bin/python ~/documents/djcode/st/sycorax/tweet.py Private Wiki Notebook >/tmp/stdout.log 2>/tmp/stderr.log
  • 16:24 still nothing
    • Exception: Could not find config.json file in directory Private Wiki Notebook - have to review that os.join call
    • tweak os.join params at 16:31
    • after various path tweaks, working at 17:25!

Tweet at 17:30 failed because twitter.TwitterError: [{u'message': u'Status is over 140 characters.', u'code': 186}] so have to fix that next...

Actually, first I appended an extra 3 chars at the end of each tweet to identify which stream it came from.

Next I'm generating a bigger output list for each stream (so far I'm caching this once, then picking a random line to tweet).

Backing off frequency to every 45min (and some will get skipped because of too-long-error) at 19:38. (Last successful was 19:35.)

Apr29, 07:34 - just truncating long tweets, not doing anything fancy....

Distraction - First World Stoic TwitterBot

Silly idea - find tweets that whine, combine with a quote from Stoicism. Tweet that as reply.

  • first cut - pick from tweets that use FirstWorldProblem (so they're not really whining, but good start)

Made account https://twitter.com/FirstWorldStoic - did a couple by hand.

Apr29: coded, in cron for every 2hrs on the x:02

May03: First World Stoic bot's account has been suspended, because of its use of reply-to. So I turned off that feature.

  • account re-activated.
  • (I should probably try just "mentioning" the original user. But that will eat up more message characters....)
    • Jul25: revert to treating it as a Reply 10% of the time

Sock Puppet

Jul24'2014 Idea: mimic groups of like-thinking people

  • account https://twitter.com/puppet_ebooks
  • follow clusters of like-thinking people, assign to cluster-specific list
  • each run, pick 1 list at random, grab last 200 tweets from that list, feed through DavidRoss' Markov Chain logic, spit out a tweet.

Realize that 200 tweets is an awfully small corpus, so change chain_length to 1.

Jul25: tweet a few by hand

  • quickly get suspended, because people I'm quoting do a lot of responding, so most tweets end up with a handle in them.
    • so replace the @ with # in the strings I feed into the machine...
      • then change it (14:26) to leave the handle 10% of the time
      • that got me suspended again in awhile, so backed that off to 0.1% for now...

WikiLogPageStorm

Sept11'2014 - Idea: Generate Tweet Storm from an already-existing WikiLog page. Recognizing that my blog-writing style is close to tweet-length. Perhaps I should accentuate that instead of regretting it. So let's play....

Anticipating wanting to skip these when I start PESOS model of copying all my tweets to the WikiLog.

  • prefix each one with w
  • first tweet would be page name/url: w0/ School Choice <http://webseitz.fluxent.com/wiki/SchoolChoice>

Grab raw SmartAscii body.

  • may just do this by hand, so can strip out any start/end stuff that doesn't seem relevant....

Strip out any links.

How handle...

  • italics/bold - probably strip
  • bullet lists (which can have multiple levels)...
    • tempting to sub-number w3a but that's crazy, more likely will ignore and strip out the * that create the bullets.

Line for each sentence? Or just each paragraph?

  • Most of my paragraphs are 1 sentence, so just stick with that
  • Truncate line for length? Or turn into multiple tweets? I think I'll just truncate.

Sept12: storm School Choice

Sept'2017 re-activating

I noticed ages back that my bots had all died.

I just tried to use the tweet-storm bits, and ran into issues over old OpenSSL. Then the old problems with being unable to update brew.

So trying to use the VirtualEnv from Flask.

  • then have to pip install python-twitter

Then get various SSL warnings

/Users/billseitz/Documents/djcode/wikiweb/lib/python2.7/site-packages/urllib3/util/ssl_.py:339: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  SNIMissingWarning
/Users/billseitz/Documents/djcode/wikiweb/lib/python2.7/site-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning

Tried pip install 'requests[security]' per this, no dice. Though, maybe that's just a warning and it's something else/later that's really failing. Yep, I think that's it. The first tweet actually went through.

The 2nd line resulted in twitter.error.TwitterError: [{u'message': u'A media id was not found.', u'code': 325}]

  • which is weird since I'm not trying to post any media...

Sept 12 Got OpenSSL upgraded, coming back to this

Ran python puppet_ebooks.py and it worked!

  • also seems like it's auto running every so often

Try python firstWorldStoic.py

num results:  15
num net results:  11
  • didn't seem to work at first. But maybe wasn't looking at it right, or there was some delay. Now seems like it's working! (Also over-rode count param to change from 15 to 100. https://twitter.com/FirstWorldStoic/with_replies
  • also noticing more of the outputs are partial sentences, where people are using "first world problem" inline rather than as a tag, so net result isn't interesting. Need to dump some raw results to update what I keep/toss...
  • but is it auto-running?

Check crontab -e

1 */3 * * * ~/documents/djcode/st/bin/python ~/documents/djcode/st/sycorax/tweet.py PrivateWikiNotebook >/tmp/stdout.log 2>/tmp/stderr.log
3 */4 * * * ~/documents/djcode/st/bin/python ~/documents/djcode/st/sycorax/firstWorldStoic.py > /tmp/stoicout.log 2>/tmp/stoicerr.log
*/6 * * * * ~/documents/djcode/st/bin/python ~/documents/djcode/st/sycorax/puppet_ebooks.py >/tmp/puppetout.log 2>/tmp/puppeterr.log
  • ah, only the puppet_ebooks runs every 6 minutes - the others run only once every few hours.

Sept13: dump FirstWorldStoic tweets ending in 'is', which gets rid of the most-frequent bad cases. There are others, but much more rare.

Also - requested my archive, should really try a bot of those.

Jul'2019 died: (2018-07-16) CoachBot Random Reminders

Aug'2020: tried/failed to revive (2020-08-09) Reviving TwitterBots

Edited:    |       |    Search Twitter for discussion

Made with flux.garden