Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[red-knot] Eagerly normalize type[] types #15272

Merged
merged 5 commits into from
Jan 7, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ def _(t: type[object]):
if issubclass(t, B):
reveal_type(t) # revealed: type[A] & type[B]
else:
reveal_type(t) # revealed: type[object] & ~type[A]
reveal_type(t) # revealed: type & ~type[A]
```

### Handling of `None`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,25 @@ class Foo(type[int]): ...
# TODO: should be `tuple[Literal[Foo], Literal[type], Literal[object]]
reveal_type(Foo.__mro__) # revealed: tuple[Literal[Foo], Unknown, Literal[object]]
```

## `@final` classes

`type[]` types are eagerly converted to class-literal types if a class decorated with `@final` is
used as the type argument. This applies to standard-library classes and user-defined classes:

```toml
[environment]
python-version = "3.10"
```

```py
from types import EllipsisType
from typing import final

@final
class Foo: ...

def _(x: type[Foo], y: type[EllipsisType]):
reveal_type(x) # revealed: Literal[Foo]
reveal_type(y) # revealed: Literal[EllipsisType]
Comment on lines +164 to +165
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is probably not acceptable? That is, Literal[X] for classes X is already a notation we are inventing; I don't think we can round-trip explicit user annotations of type[X] as Literal[X] in our type display, even when they are equivalent.

We could always display Literal[X] as type[X] in general. The downside here is that then we are conflating two types that are meaningfully different in how we handle them internally, with identical text representations.

I'm not sure what the alternative is, though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few points.

Firstly, we're far from unique in inventing our own notation for type display in some cases and using it in the output of reveal_type. E.g. take a look at the mypy output here. There have been a couple of complaints about some of the stranger symbols in mypy's reveal_type output being confusing, but I don't think it's really been a major issue at all. I can only remember one issue about it, and it's not highly upvoted.

Secondly, we're also far from unique in eagerly simplifying types that we see in user annotations and using the simplified types in our type display. We already do this elsewhere when we simplify unions (removing duplicate elements and elements that are subtypes of each other); so does pyright, to some extent.

Notwithstanding those two points, I have been thinking for a while that our current display for class-literal and function-literal types is a little confusing for users. It looks too close to something that would be valid in a type annotation in user code, and users will probably think that we're claiming that Literal[SomeClass] is valid in a user type annotation. It'll be confusing for them when we then reject them using Literal[SomeClass] in their own code.

Prior to this PR, I was wondering about Literal[<class 'SomeClass'>] and Literal[<function 'some_function'>] as an alternative display for class-literal and function-literal types. But your comments here make me wonder if we could also consider type[SomeClass(final=True)] or type[SomeClass(@final=True)] for class-literal types

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree that both displaying our own notation for some types, and simplifying user-provided types, are things we can do. It just feels a bit like crossing another line to combine those two things in such a way that we eagerly replace a spellable user-provided type with a non-spellable type display. But maybe this isn't actually a problem in practice. I'm OK with deferring that question from this PR, and at some point following up more holistically on type display. There are a lot of interesting questions here, including how we balance conciseness/readability vs clarity (to first time readers without needing to refer to docs) in the notations we choose. I won't get too deep into the details here, I'll just say that a) I kind of like the idea of representing ClassLiteral types in some way using type[] notation, since they are similar, and b) I'm not sure we should use the word "final" in that display, because I think it's too confusing in cases where we have a ClassLiteral type but for a non-final class.

```
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,8 @@ x: type = A() # error: [invalid-assignment]

```py
def f(x: type[object]):
reveal_type(x) # revealed: type[object]
# TODO: bound method types
reveal_type(x.__repr__) # revealed: Literal[__repr__]
reveal_type(x) # revealed: type
reveal_type(x.__repr__) # revealed: @Todo(instance attributes)

class A: ...

Expand Down
244 changes: 97 additions & 147 deletions crates/red_knot_python_semantic/src/types.rs

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions crates/red_knot_python_semantic/src/types/class_base.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@ pub enum ClassBase<'db> {
}

impl<'db> ClassBase<'db> {
pub const fn is_dynamic(self) -> bool {
match self {
ClassBase::Any | ClassBase::Unknown | ClassBase::Todo(_) => true,
ClassBase::Class(_) => false,
}
}

pub fn display(self, db: &'db dyn Db) -> impl std::fmt::Display + 'db {
struct Display<'db> {
base: ClassBase<'db>,
Expand Down
18 changes: 8 additions & 10 deletions crates/red_knot_python_semantic/src/types/display.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ use ruff_python_literal::escape::AsciiEscape;

use crate::types::class_base::ClassBase;
use crate::types::{
ClassLiteralType, InstanceType, IntersectionType, KnownClass, StringLiteralType,
SubclassOfType, Type, UnionType,
ClassLiteralType, InstanceType, IntersectionType, KnownClass, StringLiteralType, Type,
UnionType,
};
use crate::Db;
use rustc_hash::FxHashMap;
Expand Down Expand Up @@ -84,16 +84,14 @@ impl Display for DisplayRepresentation<'_> {
}
// TODO functions and classes should display using a fully qualified name
Type::ClassLiteral(ClassLiteralType { class }) => f.write_str(class.name(self.db)),
Type::SubclassOf(SubclassOfType {
base: ClassBase::Class(class),
}) => {
Type::SubclassOf(subclass_of_ty) => match subclass_of_ty.subclass_of() {
// Only show the bare class name here; ClassBase::display would render this as
// type[<class 'Foo'>] instead of type[Foo].
write!(f, "type[{}]", class.name(self.db))
}
Type::SubclassOf(SubclassOfType { base }) => {
write!(f, "type[{}]", base.display(self.db))
}
ClassBase::Class(class) => write!(f, "type[{}]", class.name(self.db)),
base @ (ClassBase::Any | ClassBase::Todo(_) | ClassBase::Unknown) => {
write!(f, "type[{}]", base.display(self.db))
}
},
Type::KnownInstance(known_instance) => f.write_str(known_instance.repr(self.db)),
Type::FunctionLiteral(function) => f.write_str(function.name(self.db)),
Type::Union(union) => union.display(self.db).fmt(f),
Expand Down
9 changes: 5 additions & 4 deletions crates/red_knot_python_semantic/src/types/infer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@ use crate::semantic_index::semantic_index;
use crate::semantic_index::symbol::{NodeWithScopeKind, NodeWithScopeRef, ScopeId};
use crate::semantic_index::SemanticIndex;
use crate::stdlib::builtins_module_scope;
use crate::types::class_base::ClassBase;
use crate::types::diagnostic::{
report_invalid_assignment, report_unresolved_module, TypeCheckDiagnostics, CALL_NON_CALLABLE,
CALL_POSSIBLY_UNBOUND_METHOD, CONFLICTING_DECLARATIONS, CONFLICTING_METACLASS,
Expand All @@ -65,7 +64,7 @@ use crate::types::{
typing_extensions_symbol, Boundness, CallDunderResult, Class, ClassLiteralType, FunctionType,
InstanceType, IntersectionBuilder, IntersectionType, IterationOutcome, KnownClass,
KnownFunction, KnownInstanceType, MetaclassCandidate, MetaclassErrorKind, SliceLiteralType,
Symbol, Truthiness, TupleType, Type, TypeAliasType, TypeArrayDisplay,
SubclassOfType, Symbol, Truthiness, TupleType, Type, TypeAliasType, TypeArrayDisplay,
TypeVarBoundOrConstraints, TypeVarInstance, UnionBuilder, UnionType,
};
use crate::unpack::Unpack;
Expand Down Expand Up @@ -4891,9 +4890,11 @@ impl<'db> TypeInferenceBuilder<'db> {
ast::Expr::Name(_) | ast::Expr::Attribute(_) => {
let name_ty = self.infer_expression(slice);
match name_ty {
Type::ClassLiteral(ClassLiteralType { class }) => Type::subclass_of(class),
Type::ClassLiteral(ClassLiteralType { class }) => {
SubclassOfType::from(self.db(), class)
}
Type::KnownInstance(KnownInstanceType::Any) => {
Type::subclass_of_base(ClassBase::Any)
SubclassOfType::subclass_of_any()
}
_ => todo_type!("unsupported type[X] special form"),
}
Expand Down
20 changes: 11 additions & 9 deletions crates/red_knot_python_semantic/src/types/narrow.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ use crate::semantic_index::expression::Expression;
use crate::semantic_index::symbol::{ScopeId, ScopedSymbolId, SymbolTable};
use crate::semantic_index::symbol_table;
use crate::types::{
infer_expression_types, ClassBase, ClassLiteralType, IntersectionBuilder, KnownClass,
KnownFunction, SubclassOfType, Truthiness, Type, UnionBuilder,
infer_expression_types, ClassLiteralType, IntersectionBuilder, KnownClass, KnownFunction,
SubclassOfType, Truthiness, Type, UnionBuilder,
};
use crate::Db;
use itertools::Itertools;
Expand Down Expand Up @@ -97,6 +97,11 @@ impl KnownConstraintFunction {
/// The `classinfo` argument can be a class literal, a tuple of (tuples of) class literals. PEP 604
/// union types are not yet supported. Returns `None` if the `classinfo` argument has a wrong type.
fn generate_constraint<'db>(self, db: &'db dyn Db, classinfo: Type<'db>) -> Option<Type<'db>> {
let constraint_fn = |class| match self {
KnownConstraintFunction::IsInstance => Type::instance(class),
KnownConstraintFunction::IsSubclass => SubclassOfType::from(db, class),
};

match classinfo {
Type::Tuple(tuple) => {
let mut builder = UnionBuilder::new(db);
Expand All @@ -105,13 +110,10 @@ impl KnownConstraintFunction {
}
Some(builder.build())
}
Type::ClassLiteral(ClassLiteralType { class })
| Type::SubclassOf(SubclassOfType {
base: ClassBase::Class(class),
}) => Some(match self {
KnownConstraintFunction::IsInstance => Type::instance(class),
KnownConstraintFunction::IsSubclass => Type::subclass_of(class),
}),
Type::ClassLiteral(ClassLiteralType { class }) => Some(constraint_fn(class)),
Type::SubclassOf(subclass_of_ty) => {
subclass_of_ty.subclass_of().into_class().map(constraint_fn)
}
_ => None,
}
}
Expand Down
92 changes: 92 additions & 0 deletions crates/red_knot_python_semantic/src/types/subclass_of.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
use super::{ClassBase, ClassLiteralType, Db, KnownClass, Symbol, Type};

/// A type that represents `type[C]`, i.e. the class object `C` and class objects that are subclasses of `C`.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, salsa::Update)]
pub struct SubclassOfType<'db> {
// Keep this field private, so that the only way of constructing the struct is through the `from` method.
subclass_of: ClassBase<'db>,
}

impl<'db> SubclassOfType<'db> {
/// Construct a new [`Type`] instance representing a given class object (or a given dynamic type)
/// and all possible subclasses of that class object/dynamic type.
///
/// This method does not always return a [`Type::SubclassOf`] variant.
/// If the class object is known to be a final class,
/// this method will return a [`Type::ClassLiteral`] variant; this is a more precise type.
/// If the class object is `builtins.object`, `Type::Instance(<builtins.type>)` will be returned;
/// this is no more precise, but it is exactly equivalent to `type[object]`.
///
/// The eager normalization here means that we do not need to worry elsewhere about distinguishing
/// between `@final` classes and other classes when dealing with [`Type::SubclassOf`] variants.
pub(crate) fn from(db: &'db dyn Db, subclass_of: impl Into<ClassBase<'db>>) -> Type<'db> {
let subclass_of = subclass_of.into();
match subclass_of {
ClassBase::Any | ClassBase::Unknown | ClassBase::Todo(_) => {
Type::SubclassOf(Self { subclass_of })
}
ClassBase::Class(class) => {
if class.is_final(db) {
Type::ClassLiteral(ClassLiteralType { class })
} else if class.is_known(db, KnownClass::Object) {
KnownClass::Type.to_instance(db)
} else {
Type::SubclassOf(Self { subclass_of })
}
}
}
}

/// Return a [`Type`] instance representing the type `type[Unknown]`.
pub(crate) const fn subclass_of_unknown() -> Type<'db> {
Type::SubclassOf(SubclassOfType {
subclass_of: ClassBase::Unknown,
})
}

/// Return a [`Type`] instance representing the type `type[Any]`.
pub(crate) const fn subclass_of_any() -> Type<'db> {
Type::SubclassOf(SubclassOfType {
subclass_of: ClassBase::Any,
})
}

/// Return the inner [`ClassBase`] value wrapped by this `SubclassOfType`.
pub(crate) const fn subclass_of(self) -> ClassBase<'db> {
self.subclass_of
}

pub const fn is_dynamic(self) -> bool {
// Unpack `self` so that we're forced to update this method if any more fields are added in the future.
let Self { subclass_of } = self;
subclass_of.is_dynamic()
}

pub const fn is_fully_static(self) -> bool {
!self.is_dynamic()
}

pub(crate) fn member(self, db: &'db dyn Db, name: &str) -> Symbol<'db> {
Type::from(self.subclass_of).member(db, name)
}

/// Return `true` if `self` is a subtype of `other`.
///
/// This can only return `true` if `self.subclass_of` is a [`ClassBase::Class`] variant;
/// only fully static types participate in subtyping.
pub(crate) fn is_subtype_of(self, db: &'db dyn Db, other: SubclassOfType<'db>) -> bool {
match (self.subclass_of, other.subclass_of) {
// Non-fully-static types do not participate in subtyping
(ClassBase::Any | ClassBase::Unknown | ClassBase::Todo(_), _)
| (_, ClassBase::Any | ClassBase::Unknown | ClassBase::Todo(_)) => false,

// For example, `type[bool]` describes all possible runtime subclasses of the class `bool`,
// and `type[int]` describes all possible runtime subclasses of the class `int`.
// The first set is a subset of the second set, because `bool` is itself a subclass of `int`.
(ClassBase::Class(self_class), ClassBase::Class(other_class)) => {
// N.B. The subclass relation is fully static
self_class.is_subclass_of(db, other_class)
}
}
}
}
Loading