Simplify `Configuration`

[janbuchar]

• a library may exist to make working with environment-based configuration easier (similar to pydantic-settings in Python)
• we should reconsider the current configuration priority where environment variables take precedence over arguments to the Configuration constructor
• we should consider removing support for the crawlee.json configuration file
• we should move strictly Apify-specific configuration options to the SDK (e.g., ACTOR_DEFAULT_DATASET_ID)

[B4nan]

a library may exist to make working with environment-based configuration easier (similar to pydantic-settings in Python)
we should reconsider the current configuration priority where environment variables take precedence over arguments to the Configuration constructor

To be decided later.

we should consider removing support for the crawlee.json configuration file

Let's confirm with our users (mainly but not only delivery) if they use this, likely nope so fine to remove.

we should move strictly Apify-specific configuration options to the SDK (e.g., ACTOR_DEFAULT_DATASET_ID)

Most important bit, python version already works this way.

[B4nan]

Recent discussion about the env var priorities: https://apify.slack.com/archives/C01E7CSJ2PP/p1768826609220829

In v4, we should rework this so user-provided values take precedence over env vars, but our defaults that the config class resolves now, should have lower priority than env vars.