Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All User Followings Not Returned (Missing Accounts) #236

Open
halogen857 opened this issue Jan 5, 2025 · 13 comments
Open

All User Followings Not Returned (Missing Accounts) #236

halogen857 opened this issue Jan 5, 2025 · 13 comments

Comments

@halogen857
Copy link

halogen857 commented Jan 5, 2025

Hi,

I think there's an issue fetching all the user's followings. Sometimes theres some missing for some accounts, not all. Not sure if I am doing something wrong myself.

Note: Lots of duplicates are also returned. I use a set to remove any potential duplicates.

Example Result

Notice it returns 255 but there are 256 total for account rookisaacman
Screenshot 2025-01-05 at 5 19 17 AM

Logs:

Fetching followings for: rookisaacman
Fetched 69 followings (Total so far: 69)
Fetched 117 followings (Total so far: 117)
Fetched 165 followings (Total so far: 165)
Fetched 206 followings (Total so far: 206)
Fetched 249 followings (Total so far: 249)
Fetched 255 followings (Total so far: 255)
Warning: Expected 256, but only fetched 255 followings.
Total unique followings fetched: 255

Code

from dotenv import load_dotenv
import os
from tweety import Twitter
import asyncio

# Load environment variables
load_dotenv(override=True)

auth_token = os.getenv("TWITTER_AUTH_TOKEN")
target_username = os.getenv("TWITTER_TARGET_USERNAME")

twitter_client = Twitter("session", verify=False)

async def get_all_followings():
    await twitter_client.load_auth_token(auth_token)
    print(f"Fetching followings for: {target_username}")

    followings = set()

    async for users, _ in twitter_client.iter_user_followings(
        target_username, 
        pages=10,
        wait_time=1
    ):
        followings.update(user.username for user in users)
        print(f"Fetched {len(users)} followings (Total so far: {len(followings)})")

    # Get user info to compare expected followings count
    user_info = await twitter_client.get_user_info(target_username)
    expected_count = user_info.friends_count  # Use friends_count instead of following_count

    if len(followings) < expected_count:
        print(f"Warning: Expected {expected_count}, but only fetched {len(followings)} followings.")
    else:
        print("All followings fetched successfully.")

    print(f"Total unique followings fetched: {len(followings)}")

# Run the async function
asyncio.run(get_all_followings())
@halogen857
Copy link
Author

@mahrtayyab thoughts?

@mahrtayyab
Copy link
Owner

if you are comfortable , can you please share username

@halogen857
Copy link
Author

hi @mahrtayyab, the username (target_username) that I am referring to is rookisaacman (I also mentioned that in post above).

But you can try different usernames and insert into target_username in code example from Twitter. You'll see some accounts return less followers than the count displays.

@mahrtayyab
Copy link
Owner

Are you able to see all the following list on frontend?

@mahrtayyab
Copy link
Owner

mahrtayyab commented Jan 6, 2025

image
Looks like Twitter don't us to get all users , it is returning empty objects

@halogen857
Copy link
Author

halogen857 commented Jan 6, 2025

Are you able to see all the following list on frontend?

The Twitter frontend here you mean?

Screenshot 2025-01-06 at 1 09 21 PM

No, the frontend only shows like 50 followers

@halogen857
Copy link
Author

Looks like Twitter don't us to get all users , it is returning empty objects

Is this why 1 account was missing?

@mahrtayyab
Copy link
Owner

Are you able to see all the following list on frontend?

The Twitter frontend here you mean?

Screenshot 2025-01-06 at 1 09 21 PM No, the frontend only shows like 50 followers

Yup clearly Twitter don't want us to get all followings list , I will confirm after further investigation.

@mahrtayyab
Copy link
Owner

Are you able to see all the following list on frontend?

The Twitter frontend here you mean?
Screenshot 2025-01-06 at 1 09 21 PM
No, the frontend only shows like 50 followers

Yup clearly Twitter don't want us to get all followings list , I will confirm after further investigation.

Just Like User Tweets

@halogen857
Copy link
Author

Yup clearly Twitter don't want us to get all followings list , I will confirm after further investigation.

Hm, I think your code works (see my example above)? Maybe just some accounts are not returned for some reason? Please let me know if can try code

@halogen857
Copy link
Author

@mahrtayyab any updates on this? thanks

@mahrtayyab
Copy link
Owner

@mahrtayyab any updates on this? thanks

I have to force the user to be obtained using ID (if available) which will incure a API Call. Will be added in next release

@halogen857
Copy link
Author

interesting, so it is an issue. Thank you for checking and I'm looking forward to update!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants