This is a major release to signal the end of API v1.1 functions.
Fix a problem with auth_sitrep() not correctly handling old tokens.
Since httr2 > 0.2.3, rtweet refreshes OAuth 2.0 tokens automatically.
It is no longer needed to user rtweet_oauth2(), authentication is fully handled by httr2.
Document the deprecated functions from API v1.1 to API v2: see help("rtweet-deprecated", "rtweet").
New function client_clean() to remove client's data (together with auth_clean() they leave the user cache empty).
Fix auth_sitrep() to work well with OAuth2 tokens.
Added new function tweet_quoted() to search who tweets quoting a tweet.
New tweet_post() and tweet_delete() to post and delete tweets to work with the free product.
Fix problems with changes on is.atomic(NULL).
New client_* functions to save, set and use a client app using Twitter API
via the new authentication mechanism rtweet_oauth2() (see below).
It has a helper function for the scopes the client is allowed to do set_scopes().
Endpoint have different scope requirements.
New authentication mechanism rtweet_oauth2() required by some endpoints.
Only valid for 2 hours until it is automatically renewed, but it is left to
the user to save it every time it is renewed.
New endpoints to retrieve data from twitter, no action is performed (no blocking, muting, posting new tweets...):
New list endpoints: list_expansions(), list_fields(), list_followers(),
list_get(), list_members(), list_membership(), list_tweets().
New tweet endpoints: tweet_get(), tweet_liking_users(), tweet_retweeted_by().
New user endpoints: user_blocked(),user_bookmarks(),user_by_username(),
user_following(), user_followers(), user_liked_tweets(),
user_list_follows(), user_lists(), user_mentions(), user_search(),
user_self(), user_timeline(), user_tweets().
New statistic endpoints: tweet_counts_recent(), tweet_counts_all().
Small breaking change: expansions and fields arguments now use NA for all and NULL for none.
New functions to set expansions and fields for the new functions: set_expansions(), set_fields().
Fixed a bug that prevented auth_setup_default() to work (#756 and #744)
Fixed a bug so that the streaming functions use the current token.
search_fullarchive and search_30day (#720).auth_sitrep() to work in some cases.rbind and subsetting methods ([) for rtweet new classes.clean_tweets() for cleaning entities from the text of tweets (#721).entity().ids() from multiple classes and print nicer in post_tweet() (#452).post_tweet() now invisible returns the tweet information.filtered_stream() and sample_stream(), including adding rules
stream_add_rules() and stream_rm_rules().lookup_tweets().tweets_with_users and users_with_tweets because Twitmo depends on them.Fixed issue with .Rbuilignore and vignettes
Reduced fixtures sizes by limiting the page size to the number of requests if it is smaller than the default page size.
Data returned by rtweet is nested and uses the same names provided by the Twitter API. It doesn't compute or add new columns as it did previously.
emojis, langs and stopwordslangs data are no longer provided by rtweet.
get_friends() and get_followers() return similar formatted output with
two columns "from_id" and "to_id" (#308, @alexpghayes).
All paginated functions that don't return tweets now use a consistent
pagination interface. They all store the "next cursor" in an rtweet_cursor
attribute, which will be automatically retrieved when you use the cursor
argument.
Functions that return tweets (e.g. get_favorites(), get_my_timeline(),
get_timeline(), get_mentions(), lists_statuses() and search_tweets())
now expose a consistent pagination interface. They all support max_id and
since_id to find earlier and later tweets respectively, as well as
retryonratelimit to wait as long as needed when rate limited (#510).
suggested_slugs(), suggested_users(), suggested_users_all() have been
removed as they stopped working when Twitter remove the suggested users
endpoint in June 2019 (https://twittercommunity.com/t/124732).
parse = FALSE always means return the raw "JSON". Previously some functions
(e.g. my_friendships()) would return the raw HTTP response instead (#504).
rtweet no longer re-exports the magrittr pipe %>%; if you want to continue
using it, you'll need to library(magrittr) or library(dplyr) (#522).
The authentication system has been rewritten, check the following section.
lookup_collections() and get_collections() has been hard deprecated
because the underlying Twitter API has been deprecated.
previous_cursor() has been hard deprecated. It could only be used with
lists_memberships() and it has been dropped in favour of making regular
pagination better.
tweet_shot() has been hard deprecated as the screenshots do not have the
tweet. It might come back with webshot2 (#458).
The home argument to get_timeline() has been deprecated. You can only
retrieve the home timeline for the logged in user, and that's the job of
get_my_timeline() (#550).
Due to the changes on rtweet data format, all the functions related to
flattening the data (write_as_csv(), save_as_csv() flatten(),
unflatten(), read_twitter_csv()) are deprecated.
Users should decide how to flatten the nested structure of the data.
lookup_statuses() has been deprecated in favour of lookup_tweets().
as_userid() has been deprecated since in case of ambiguity the default is
to assume a numeric string is a user id (#520). All functions now use a
single user_type() function so behaviour is identical for all rtweet
functions.
get_timelines() has been deprecated since it does that same thing as
get_timeline() (#509).
stream_tweets2() has been deprecated in favour of stream_tweets().
rtweet's authentication system has been completely written. It is now based
around three authentication options: rtweet_user(), rtweet_app(), and
rtweet_bot(). Authentication no longer touches ~/.Renviron file; instead
auth_save() and auth_as() allow you to explicitly save and load
authentication mechanisms from a system config directory. See vignette("auth")
for more details.
The httpuv package is now only suggested, since it's only needed for interactive auth, and you'll be prompted to install it when needed.
bearer_token() has been deprecated in favour of rtweet_app(), which takes
the bearer token found in your Twitter developer portal. invalidate_bearer()
has been deprecated since this is something you should do yourself in the
Twitter developer portal.
create_token() has been deprecated in favour of the combination of
rtweet_user()/rtweet_bot()/rtweet_app() + auth_as() + auth_save().
get_token() and get_tokens() have been deprecated in favour of
auth_get() and auth_list().
auth_as() accepts path to an authentication to make it easier to use
authentications outside a user account (#602, @maelle)
auth_setup_default() will not only authenticate and save but use the
default token.
The new auth_sitrep() helps reports the different authentications of the user
Update to new rOpenSci Code of Conduct: https://ropensci.org/code-of-conduct/
lookup_users() and search_users() now returns a data frame containing
all information about each user (not their latest tweet). If you want to get
that data you can use tweets_data().
rtweet 1.0.0 implements a consistent strategy for handling rate limits.
By default, if a paginated function (i.e. a rtweet function that performs
multiple calls to the twitter API) is rate-limited it will return all results
received up to that point, along with a warning telling you how to get more
results. Alternatively, if you want to automatically wait until the
rate-limit is reset, you can set retryratelimit = TRUE.
The default value of retryonratelimit comes from the option
rtweet.retryonratelimit so you can globally set it to TRUE if desired
(#173).
All functions that perform multiple requests on your behalf now display
a progress bar so you know what's happening. If you don't want it, you can
turn it off with verbose = FALSE (#518).
Banned or protected accounts now trigger a warning instead of an error, but if data from other users is requested it is not served by the API and returned as NA (#590, @simonheb).
Added support for posting alt-text metadata with images tweeted with status
updated via post_tweet(). (#425, @hrbrmstr)
stream_tweets() has been overhauled to only write valid data. This obsoletes
all previous strategy to clean up bad data after the fact (#350, #356).
The maintainer changed.
New user_block() and user_unblock() to block and unblock users (#593,
@simonheb).
The new tweet_threading function is now faster and more reliable (#305,
#693, @charliejhadley).
Fields withheld_scope, withheld_copyright, withheld_in_countries are
now correctly parsed (#647, @alexpghayes).
Functions like search_tweets(), lookup_statuses() and others return the
appropriate date time format for the right columns (created_at mostly)
(#653, #657, #660, @alexpghayes, @RickPat).
Premium/sandbox environments are supported in search_fullarchive() and
search_30day() (#578, #713).
The vignette must be pre-computed before submission (#609, @maelle).
bearer_token() docs.list_statuses() endpointnetwork_data() and network_graph()lookup_coords() now requires a Google Maps API key. It will be stored for
easy future use once supplied.bearer_token() option for access to more generous rate limits.create_token() when using browse-based authentication
method.post_list(), which now allows users
to create and populate lists as well as delete lists on behalf of one's own
Twitter account.lists_memberships() and now scrolls through multiple pages of results to
automate collection of larger numbers of lists.create_token() which allows for creation of token
non-interactive sessions via accepting inputs for consumer key, consumer
secret (always required), oauth key, and oauth secret (optional, if supplied
then non-browser sign method is used).ts_*() functions now offer a tz (timezone) argument, allowing users to
more easily print and plot in non-UTC time.destroy_id argument in post_tweet()join_rtweet(), which omitted users who didn't have
available tweets.all_suggested_users(), which automates the collection of Twitter's
suggested users data.save_as_csv(), including addition of new
prep_as_csv() as convenience function for flattening Twitter data frames.save_as_csv() with improved flattening and ID-preserving
saving methods. The function now saves a single [joined] data set as
well.get_favorites() and in several lists_*()
functions.stream_tweets()stream_tweets2() function for more robust streaming
method. Streams JSON files to directory and reconnects following
premature disruptions.get_timeline(), get_favorites(), get_friends(), and
get_followers() now accept vectors of length > 1.users_data()stream_data(), designed to parse files that cannot
wholely fit into memory. stream_data() can now work in parallel as well.post_status() function has been fixed and can now be used to
upload media.mentions_screen_name may consist of
4 screen names).flatten() function.
Exporting functions such as save_as_csv will apply flatten by default.coords_coords,
geo_coords, and bbox_coords bounding box. The first two come in
pairs of coords (a list column) and bbox_coords comes with 8
values (longX4 latX4). This should allow users to maximize returns
on
geo-location data.plain_tweets() added for textual analysis.ts_plot() with improved time-aggregating method. Now a
wrapper around ts_data(), deprecating ts_filter.mutate_coords() and retryonratelimit.ts_plot(), ts_filter()
and more!post_. This was done to clearly distinguish write functions from
retrieval functions.ts_plot() function is now more robust with more adaptive
characteristics for variations in the number of filters, the method
of distinguishing lines, the position of the legend, and the
aesthetics of the themes.ts_filter() function which allows users to convert Twitter
data into a time series-like data frame. Users may also provide
filtering rules with which ts_filter() will subset the data as it
converts it to multiple time series, which it then outputs as a
long-form (tidy) data frame.search_tweets now includes retryonratelimit argument to
allow for searches requesting more than 18,000 tweets. This
automates what was previously possible through use of max_id.stream_tweets.parse.piper functionts_plot to enable different filtered time series and
an aesthetic overhaul of the plot function as well.as_double argument to provide flexibility in handling
id variables (as_double provides performance boost but can create
problems when printing and saving, depending on format). By default
functions will return IDs as character vectors.clean_tweets argument provided to allow user more control over
encoding and handling of non-ascii characters.search_users and implemented several
improvements to stream_tweets and plot_ts.twitter_tokens, twitter_token, tokens, or token, rtweet
will find it.search_tweets and stream_tweeetsinclude_retweets arg added to search_tweets() function.user_id class changed to double when parsed. double is significantly
faster and consumes less space. it's also capable of handling the length of
id scalars, so the only downside is truncated printing.stream_tweets()stream_tweets() function. By default,
the streaming query argument, q, is now set to an empty string,
q = "", which returns a random sample of all Tweets
(pretty cool, right?).post_tweet() function. Users can now post tweets from their R console.get_favorites() functionAdded lookup_statuses() function, which is the counterpart to
lookup_users(). Supply a vector of status IDs and return tweet data
for each status. lookup_statuses() is particularly powerful when
combined with other methods designed to collect older Tweets. Early
experiments with doing this all through R have turned out surprisingly
well, but packaging it in a way that makes it easy to do on other
machines is unlikely to happen in the short term.
Removed dplyr dependencies. Everyone should install and use dplyr,
but for sake of parsimony, it's been removed from rtweet.
Continued development of S4 classes and methods. Given removal of dplyr dependencies, I've started to integrate print/show methods that will limit the number of rows (and width of columns) when printed. Given the amount of data returned in a relatively short period of time, printing entire data frames quickly becomes headache-inducing.
Added new trends functions. Find what trending locations are
available with trends_available() and/or search for trends
worldwide or by geographical location using get_trends().
Stability improvements including integration with Travis CI and code analysis via codecov. Token encryption method also means API testing conducted on multiple machines and systems.
search_users() function! Search for users by keyword,
name, or interest and return data on the first 1000 hits.Output for search_tweets(), stream_tweets(), and
get_timeline() now consists of tweets data and contains users data
attribute.
Output for lookup_users() now consists of users data and contains
tweets data attribute.
To access users data from a tweets object or vice-versa, use
users_data() and tweets_data() functions on objects output by major
rtweet retrieval functions.
Updated testthat tests
Output for get_friends() and get_followers() is now a tibble
of "ids". To retrieve next cursor value, use new next_cursor()
function.
Major stability improvements via testthat tests for every major function.
Since previous CRAN release, numerous new features and improvements to functions returning tweets, user data, and ids.
Search function now optimized to return more tweets per search.
Numerous improvements to stability, error checks, and namespace management.
Improvements to get_friends and get_followers. Returns list
with value (next_cursor) used for next page of results. When
this value is 0, all results have been returned.
Functions get_friends and get_followers now return the list
of user ids as a tibble data table, which makes the print out much
cleaner.
Improved scrolling methods such that search_tweets and
get_timeline should return a lot more now
Added parser function to return status (tweets) AND user (users)
data frames when available. As a result, the parsed output for some
functions now comes as a list containing two data frames.
Added get_timeline function that returns tweets from selected user
Added vignettes covering tokens and search tweets
Fixed issue with count argument in search and user functions
Fixed parsing issue for return objects with omitted variables
Added clean_tweets convenience function for text analysis
More examples included in documentation.
Added recode_error argument to get_friends function. This is
especially useful for tracking networks over time.
Further integrated ROAuth methods/objects to increase
compatibility with twitteR authorization procedures.
Improved token checking procedures.
Added NEWS.md file
Added key features and more descriptions to README.md.
There are now two stable parse (convert json obj to data frame)
types. For user objects (e.g., output of lookup_users), there
is parse_user. For tweet objects (e.g., output of search_tweets
or stream_tweets), there is parse_tweets.
New parse functions are now exported, so they should available for use with compatible Twitter packages or user-defined API request operations.
More parsing improvements
Added format_date function
Various stability improvements