oauth2client/contrib/_metadata.py does not retry

[porridge]

GCE metadata server not only is not 100% reliable, but even does not have an SLA!
This means users of oauth2client.contrib.gce.AppAssertionCredentials (for example) are affected by transient "connection refused" errors, such as this:

File "/usr/local/zync/common/zync/gcp.py", line 154, in _auth_api credentials.refresh(http) 
File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 560, in refresh self._refresh(http.request) 
File "/usr/local/lib/python2.7/dist-packages/oauth2client/contrib/gce.py", line 134, in _refresh self._retrieve_info(http_request) 
File "/usr/local/lib/python2.7/dist-packages/oauth2client/contrib/gce.py", line 115, in _retrieve_info service_account=self.service_account_email or 'default') 
File "/usr/local/lib/python2.7/dist-packages/oauth2client/contrib/_metadata.py", line 99, in get_service_account_info recursive=True) 
File "/usr/local/lib/python2.7/dist-packages/oauth2client/contrib/_metadata.py", line 61, in get headers=METADATA_HEADERS 
File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1569, in request (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey) 
File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1316, in _request (response, content) = self._conn_request(conn, request_uri, method, body, headers) 
File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 1251, in _conn_request conn.connect() 
File "/usr/lib/python2.7/dist-packages/httplib2/__init__.py", line 912, in connect raise socket.error, msg 
error: [Errno 111] Connection refused        

This code should do retries with exponential backoff.

[theacodes]

I agree with this in spirit, but I'm apprehensive about doing this before #128 is done. Once we switch to urllib3, we'll get connection retries for free (yay!).

[tcroiset]

Hi,
Any update on that?
We access datastore through gcloud and face this issue quite often.
Indeed #128 looks great but I'm afraid it won't be available soon.

[theacodes]

@tcroiset I'm working on this now, and pending code reviews I hope to have 3.0.0 out by mid-october.

[tcroiset]

Hi,
I also regularly see this error (while saving an entity to datastore, using gcloud-python):

File "/usr/local/lib/python2.7/dist-packages/oauth2client/transport.py", line 153, in new_request
credentials._refresh(orig_request_method)
File "/usr/local/lib/python2.7/dist-packages/oauth2client/contrib/gce.py", line 138, in _refresh
raise client.HttpAccessTokenRefreshError(str(e))
HttpAccessTokenRefreshError: Unable to find the server at metadata.google.internal

Do you think the change you're working on will handle that too or it's a different issue?

[jacobsa]

Sorry, I thought that was a more centralized project for Google's wider oauth efforts. I've filed golang/oauth2#203.

[theacodes]

Thank you for creating this issue, however, this project is deprecatedand we will only be addressing critical security issues. You can read moreabout this deprecation here.

If you need support or help using this library, we recommend that you ask yourquestion on StackOverflow.

If you still think this issue is relevant and should be addressed, pleasecomment and let us know!