BaseSettings doesn't respect population by field name if env is present

[RobertCraigie]

Checks

• I added a descriptive title to this issue
• I have searched (google, github) for similar issues and couldn't find anything
• I have read and followed the docs and still think this is a bug

Bug

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.8.2
            pydantic compiled: False
                 install path: /usr/local/lib/python3.8/site-packages/pydantic
               python version: 3.8.6 (default, Nov 20 2020, 23:57:10)  [Clang 12.0.0 (clang-1200.0.32.27)]
                     platform: macOS-11.2.1-x86_64-i386-64bit
     optional deps. installed: ['typing-extensions']

When the following are true

• Config.allow_population_by_field_name is True
• Environment variable is present for a field
• field has an alias
• field name is passed to init

an extra fields not permitted error will be raised, however the same error is not raised if the field alias is passed to init instead of the field name

import os
from pydantic import BaseSettings, Field


class Config(BaseSettings):
    user_name: str = Field(alias='userName', env='CONFIG_USER_NAME')

    class Config:
        allow_population_by_field_name = True


os.environ.pop('CONFIG_USER_NAME', None)

c = Config(userName='alice')
print(c)  # alice

c = Config(user_name='mary')
print(c)  # mary


os.environ['CONFIG_USER_NAME'] = 'bob'

c = Config()
print(c)  # bob

c = Config(userName='lisa')
print(c)  # lisa

c = Config(user_name='chris')  # error, extra fields not permitted
print(c)  # should be "chris"

[PrettyWood]

Hi @RobertCraigie
The problem behind the scene is this one

from pydantic import BaseModel, Extra, Field


class M(BaseModel, allow_population_by_field_name=True, extra=Extra.forbid):
    x: str = Field(alias='y')

data = {'x': 'pikaX'}
assert M(**data).x == 'pikaX'  # works

data = {'y': 'pikaY'}
assert M(**data).x == 'pikaY'  # works

data = {'x': 'pikaX', 'y': 'pikaY'}  # raises an error

BaseSettings uses has by default extra = Extra.forbid hence the error. If you set Extra.allow it will override the default value but will also add extra values that may not be needed.

[RobertCraigie]

Ah I understand the issue, I would still like to see this use case supported through a config option or similar if that would be possible?

There is an also an issue with setting Extra.allow in this situation as it skips validation although I haven't looked through the docs for this case so if it is a documented edge case, I apologise

from pydantic import BaseModel, Extra, Field


class M(BaseModel, allow_population_by_field_name=True, extra=Extra.allow):
    x: int = Field(alias='y')

data = {'x': 1}
assert M(**data).x == 1  # works

data = {'y': 2}
assert M(**data).x == 2  # works

data = {'x': 'foo'}  # raises an error as it should

data = {'x': 'foo', 'y': 2}  # works but sets x to 'foo'
assert M(**data).x == 'foo'

[chbndrhnns]

This issue persists in pydantic v2. It seems setting populate_by_name to True helps:

import pytest
from pydantic_settings import BaseSettings, SettingsConfigDict
import pydantic as v2


class V2(BaseSettings):
    log_level: str = v2.Field("INFO", validation_alias="log")

    model_config = SettingsConfigDict(
        populate_by_name=True,
    )


@pytest.mark.parametrize(
    "arg",
    [
        "log_level",
        "log",
    ],
)
def test_v2(arg):
    data = {arg: "ERROR"}
    assert V2(**data).log_level == "ERROR"

[hramezani]

Closing this because we have more detailed discussion in #180