-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bump PMAT version to 0.9.3, simplify calculate_bet_amount
for KnownOutcomeAgent
#51
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,14 +7,13 @@ | |
from prediction_market_agent_tooling.deploy.constants import OWNER_KEY | ||
from prediction_market_agent_tooling.gtypes import SecretStr, private_key_type | ||
from prediction_market_agent_tooling.markets.agent_market import AgentMarket | ||
from prediction_market_agent_tooling.markets.data_models import BetAmount, Currency | ||
from prediction_market_agent_tooling.markets.data_models import BetAmount | ||
from prediction_market_agent_tooling.markets.markets import MarketType | ||
from prediction_market_agent_tooling.markets.omen.omen import OmenAgentMarket | ||
from prediction_market_agent_tooling.tools.utils import ( | ||
check_not_none, | ||
get_current_git_commit_sha, | ||
get_current_git_url, | ||
should_not_happen, | ||
) | ||
|
||
from prediction_market_agent.agents.known_outcome_agent.known_outcome_agent import ( | ||
|
@@ -28,14 +27,19 @@ def market_is_saturated(market: AgentMarket) -> bool: | |
|
||
|
||
class DeployableKnownOutcomeAgent(DeployableAgent): | ||
model = "gpt-4-1106-preview" | ||
model = "gpt-4-turbo-preview" | ||
|
||
def load(self) -> None: | ||
self.markets_with_known_outcomes: dict[str, Result] = {} | ||
|
||
def pick_markets(self, markets: list[AgentMarket]) -> list[AgentMarket]: | ||
picked_markets: list[AgentMarket] = [] | ||
for market in markets: | ||
if not isinstance(market, OmenAgentMarket): | ||
raise NotImplementedError( | ||
"This agent only supports predictions on Omen markets" | ||
) | ||
|
||
print(f"Looking at market {market.id=} {market.question=}") | ||
|
||
# Assume very high probability markets are already known, and have | ||
|
@@ -45,6 +49,10 @@ def pick_markets(self, markets: list[AgentMarket]) -> list[AgentMarket]: | |
print( | ||
f"Skipping market {market.id=} {market.question=}, because it is already saturated." | ||
) | ||
elif market.get_liquidity_in_xdai() < 5: | ||
print( | ||
f"Skipping market {market.id=} {market.question=}, because it has insufficient liquidity." | ||
) | ||
else: | ||
picked_markets.append(market) | ||
|
||
|
@@ -83,17 +91,7 @@ def answer_binary_market(self, market: AgentMarket) -> bool | None: | |
|
||
def calculate_bet_amount(self, answer: bool, market: AgentMarket) -> BetAmount: | ||
if isinstance(market, OmenAgentMarket): | ||
if market.currency != Currency.xDai: | ||
should_not_happen() | ||
return BetAmount( | ||
# On markets without liquidity, bet just a small amount for benchmarking. | ||
amount=( | ||
Decimal(1.0) | ||
if market.get_liquidity_in_xdai() > 5 | ||
else market.get_tiny_bet_amount().amount | ||
), | ||
currency=Currency.xDai, | ||
) | ||
return BetAmount(amount=(Decimal(1.0)), currency=market.currency) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I know we now filtering for markets with liquidity, but people can still create markets with for example, $0.01 in liquidity and then it doesn't make much sense to me to bet more than the minimal bet. wdyt? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good point. In that case, I think it makes sense to filter the market at the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't you want to keep them for benchmark purposes? If not, okay by me, just that
Should it be with the opposite sign, probably? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oops! 😆 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I'm not too concerned about that, more about trying to get the markets that should be p_yes/no==0.95 to that point |
||
else: | ||
raise NotImplementedError("This agent only supports xDai markets") | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just noticed this as well, this can point to a different model in future without notice, is that desirable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm good question. Maybe in general not, but I reckon for this case it's good - my reasoning being that otherwise over time the agent will get less and less 'up to date with current events' which I would assume is bad for making future predictions.