How is the namespace of the module object implemented

Python namespace tutorial

In this Python tutorial, we will learn about namespace in python. Also, we will check:

Introduction to Python namespace

Local Namespace in Python

In this example, I have defined a function vehicle, and the car is the local namespace that is inside the function vehicle.

Below screenshot shows the output:

Python Global Namespace

Below screenshot shows the output:

Built-in Namespace in Python

In this example, I am not defining any function print is a built-in namespace in python.

Below screenshot shows the output:

Python namespace to dictionary

Here, we can see how to convert argparse.namespace() object into a dictionary in python. The argparse module is easy to write a command-line interface, to import argparse module.

In this example, I have imported argparse module, item() returns all the key-value pairs the variables are converted to the dictionary the namespace variables are converted into a dictionary using converted_dict = vars(namespace).

Below screenshot shows the output:

Python namespace class

Below screenshots shows the output:

In this output, we can see the attributes assigned to args.

Python namespace import

Now, we can see the different types of import statements in python.

The import statement

To get access to code from modules by importing files using import.

The from statement

The from statement is used to import specific attributes from the module into the present namespace.

The from import* statement

Here, we can import any module name.

Python namespace and scope

When only a variable is available from inside the function region that is called scope. In this example, variable multiplication is not available for outside functions it’s only available for the inside function.

Below screenshots shows the output:

We can see the correct output because the print statement is inside the function.

Here, we can see if we write the print statement outside the function then we will get an error. So, we have to follow the print statement inside the function in order to get the correct output.

You may like the following python tutorials:

Here, we learned about Python namespace.

Entrepreneur, Founder, Author, Blogger, Trainer, and more. Check out my profile.

What are Python namespaces all about

I have just started learning Python & have come across «namespaces» concept in Python. While I got the jist of what it is, but am unable to appreciate the gravity of this concept.

Some browsing on the net revealed that one of the reasons going against PHP is that it has no native support for namespaces.

Could someone explain how to use namespaces & how this feature makes programming better (not just in Python, as I assume namespaces in not a concept limited to a particular language).

I am predominantly coming from Java and C programming backgrounds.

6 Answers 6

Trending sort

Trending sort is based off of the default sorting method — by highest score — but it boosts votes that have happened recently, helping to surface more up-to-date answers.

It falls back to sorting by highest score if no posts are trending.

Switch to Trending sort

Namespace is a way to implement scope.

In Java (or C) the compiler determines where a variable is visible through static scope analysis.

In C, scope is either the body of a function or it’s global or it’s external. The compiler reasons this out for you and resolves each variable name based on scope rules. External names are resolved by the linker after all the modules are compiled.

In Java, scope is the body of a method function, or all the methods of a class. Some class names have a module-level scope, also. Again, the compiler figures this out at compile time and resolves each name based on the scope rules.

In Python, each package, module, class, function and method function owns a «namespace» in which variable names are resolved. Plus there’s a global namespace that’s used if the name isn’t in the local namespace.

Each variable name is checked in the local namespace (the body of the function, the module, etc.), and then checked in the global namespace.

Variables are generally created only in a local namespace. The global and nonlocal statements can create variables in other than the local namespace.

When a function, method function, module or package is evaluated (that is, starts execution) a namespace is created. Think of it as an «evaluation context». When a function or method function, etc., finishes execution, the namespace is dropped. The variables are dropped. The objects may be dropped, also.

9. Classes¶

Classes provide a means of bundling data and functionality together. Creating a new class creates a new type of object, allowing new instances of that type to be made. Each class instance can have attributes attached to it for maintaining its state. Class instances can also have methods (defined by its class) for modifying its state.

Compared with other programming languages, Python’s class mechanism adds classes with a minimum of new syntax and semantics. It is a mixture of the class mechanisms found in C++ and Modula-3. Python classes provide all the standard features of Object Oriented Programming: the class inheritance mechanism allows multiple base classes, a derived class can override any methods of its base class or classes, and a method can call the method of a base class with the same name. Objects can contain arbitrary amounts and kinds of data. As is true for modules, classes partake of the dynamic nature of Python: they are created at runtime, and can be modified further after creation.

In C++ terminology, normally class members (including the data members) are public (except see below Private Variables ), and all member functions are virtual. As in Modula-3, there are no shorthands for referencing the object’s members from its methods: the method function is declared with an explicit first argument representing the object, which is provided implicitly by the call. As in Smalltalk, classes themselves are objects. This provides semantics for importing and renaming. Unlike C++ and Modula-3, built-in types can be used as base classes for extension by the user. Also, like in C++, most built-in operators with special syntax (arithmetic operators, subscripting etc.) can be redefined for class instances.

(Lacking universally accepted terminology to talk about classes, I will make occasional use of Smalltalk and C++ terms. I would use Modula-3 terms, since its object-oriented semantics are closer to those of Python than C++, but I expect that few readers have heard of it.)

9.1. A Word About Names and Objects¶

Objects have individuality, and multiple names (in multiple scopes) can be bound to the same object. This is known as aliasing in other languages. This is usually not appreciated on a first glance at Python, and can be safely ignored when dealing with immutable basic types (numbers, strings, tuples). However, aliasing has a possibly surprising effect on the semantics of Python code involving mutable objects such as lists, dictionaries, and most other types. This is usually used to the benefit of the program, since aliases behave like pointers in some respects. For example, passing an object is cheap since only a pointer is passed by the implementation; and if a function modifies an object passed as an argument, the caller will see the change — this eliminates the need for two different argument passing mechanisms as in Pascal.

9.2. Python Scopes and Namespaces¶

Before introducing classes, I first have to tell you something about Python’s scope rules. Class definitions play some neat tricks with namespaces, and you need to know how scopes and namespaces work to fully understand what’s going on. Incidentally, knowledge about this subject is useful for any advanced Python programmer.

Let’s begin with some definitions.

The local namespace for a function is created when the function is called, and deleted when the function returns or raises an exception that is not handled within the function. (Actually, forgetting would be a better way to describe what actually happens.) Of course, recursive invocations each have their own local namespace.

A scope is a textual region of a Python program where a namespace is directly accessible. “Directly accessible” here means that an unqualified reference to a name attempts to find the name in the namespace.

Although scopes are determined statically, they are used dynamically. At any time during execution, there are 3 or 4 nested scopes whose namespaces are directly accessible:

the innermost scope, which is searched first, contains the local names

the scopes of any enclosing functions, which are searched starting with the nearest enclosing scope, contains non-local, but also non-global names

the next-to-last scope contains the current module’s global names

the outermost scope (searched last) is the namespace containing built-in names

If a name is declared global, then all references and assignments go directly to the middle scope containing the module’s global names. To rebind variables found outside of the innermost scope, the nonlocal statement can be used; if not declared nonlocal, those variables are read-only (an attempt to write to such a variable will simply create a new local variable in the innermost scope, leaving the identically named outer variable unchanged).

Usually, the local scope references the local names of the (textually) current function. Outside functions, the local scope references the same namespace as the global scope: the module’s namespace. Class definitions place yet another namespace in the local scope.

It is important to realize that scopes are determined textually: the global scope of a function defined in a module is that module’s namespace, no matter from where or by what alias the function is called. On the other hand, the actual search for names is done dynamically, at run time — however, the language definition is evolving towards static name resolution, at “compile” time, so don’t rely on dynamic name resolution! (In fact, local variables are already determined statically.)

A special quirk of Python is that – if no global or nonlocal statement is in effect – assignments to names always go into the innermost scope. Assignments do not copy data — they just bind names to objects. The same is true for deletions: the statement del x removes the binding of x from the namespace referenced by the local scope. In fact, all operations that introduce new names use the local scope: in particular, import statements and function definitions bind the module or function name in the local scope.

The global statement can be used to indicate that particular variables live in the global scope and should be rebound there; the nonlocal statement indicates that particular variables live in an enclosing scope and should be rebound there.

9.2.1. Scopes and Namespaces Example¶

This is an example demonstrating how to reference the different scopes and namespaces, and how global and nonlocal affect variable binding:

The output of the example code is:

Note how the local assignment (which is default) didn’t change scope_test‘s binding of spam. The nonlocal assignment changed scope_test‘s binding of spam, and the global assignment changed the module-level binding.

You can also see that there was no previous binding for spam before the global assignment.

9.3. A First Look at Classes¶

Classes introduce a little bit of new syntax, three new object types, and some new semantics.

9.3.1. Class Definition Syntax¶

The simplest form of class definition looks like this:

Class definitions, like function definitions ( def statements) must be executed before they have any effect. (You could conceivably place a class definition in a branch of an if statement, or inside a function.)

In practice, the statements inside a class definition will usually be function definitions, but other statements are allowed, and sometimes useful — we’ll come back to this later. The function definitions inside a class normally have a peculiar form of argument list, dictated by the calling conventions for methods — again, this is explained later.

When a class definition is entered, a new namespace is created, and used as the local scope — thus, all assignments to local variables go into this new namespace. In particular, function definitions bind the name of the new function here.

When a class definition is left normally (via the end), a class object is created. This is basically a wrapper around the contents of the namespace created by the class definition; we’ll learn more about class objects in the next section. The original local scope (the one in effect just before the class definition was entered) is reinstated, and the class object is bound here to the class name given in the class definition header ( ClassName in the example).

9.3.2. Class Objects¶

Class objects support two kinds of operations: attribute references and instantiation.

Class instantiation uses function notation. Just pretend that the class object is a parameterless function that returns a new instance of the class. For example (assuming the above class):

When a class defines an __init__() method, class instantiation automatically invokes __init__() for the newly created class instance. So in this example, a new, initialized instance can be obtained by:

9.3.3. Instance Objects¶

Now what can we do with instance objects? The only operations understood by instance objects are attribute references. There are two kinds of valid attribute names: data attributes and methods.

The other kind of instance attribute reference is a method. A method is a function that “belongs to” an object. (In Python, the term method is not unique to class instances: other object types can have methods as well. For example, list objects have methods called append, insert, remove, sort, and so on. However, in the following discussion, we’ll use the term method exclusively to mean methods of class instance objects, unless explicitly stated otherwise.)

Valid method names of an instance object depend on its class. By definition, all attributes of a class that are function objects define corresponding methods of its instances. So in our example, x.f is a valid method reference, since MyClass.f is a function, but x.i is not, since MyClass.i is not. But x.f is not the same thing as MyClass.f — it is a method object, not a function object.

9.3.4. Method Objects¶

Usually, a method is called right after it is bound:

will continue to print hello world until the end of time.

What exactly happens when a method is called? You may have noticed that x.f() was called without an argument above, even though the function definition for f() specified an argument. What happened to the argument? Surely Python raises an exception when a function that requires an argument is called without any — even if the argument isn’t actually used…

If you still don’t understand how methods work, a look at the implementation can perhaps clarify matters. When a non-data attribute of an instance is referenced, the instance’s class is searched. If the name denotes a valid class attribute that is a function object, a method object is created by packing (pointers to) the instance object and the function object just found together in an abstract object: this is the method object. When the method object is called with an argument list, a new argument list is constructed from the instance object and the argument list, and the function object is called with this new argument list.

9.3.5. Class and Instance Variables¶

Generally speaking, instance variables are for data unique to each instance and class variables are for attributes and methods shared by all instances of the class:

Correct design of the class should use an instance variable instead:

Python Enhancement Proposals

PEP 420 – Implicit Namespace Packages

Abstract

Namespace packages are a mechanism for splitting a single Python package across multiple directories on disk. In current Python versions, an algorithm to compute the packages __path__ must be formulated. With the enhancement proposed here, the import machinery itself will construct the list of directories that make up the package. This PEP builds upon previous work, documented in PEP 382 and PEP 402. Those PEPs have since been rejected in favor of this one. An implementation of this PEP is at [1].

Terminology

Within this PEP:

This PEP defines a new type of package, the “namespace package”.

Namespace packages today

Python currently provides pkgutil.extend_path to denote a package as a namespace package. The recommended way of using it is to put:

.pkg which allows declaration of additional portions.

setuptools provides a similar function named pkg_resources.declare_namespace that is used in the form:

See PEP 402’s “The Problem” section for additional motivations for namespace packages. Note that PEP 402 has been rejected, but the motivating use cases are still valid.

Rationale

The current imperative approach to namespace packages has led to multiple slightly-incompatible mechanisms for providing namespace packages. For example, pkgutil supports *.pkg files; setuptools doesn’t. Likewise, setuptools supports inspecting zip files, and supports adding portions to its _namespace_packages variable, whereas pkgutil doesn’t.

Specification

Regular packages will continue to have an __init__.py and will reside in a single directory.

During import processing, the import machinery will continue to iterate over each directory in the parent path as it does in Python 3.2. While looking for a module or package named “foo”, for each directory in the parent path:

If the scan completes without returning a module or package, and at least one directory was recorded, then a namespace package is created. The new namespace package:

Note that if “import foo” is executed and “foo” is found as a namespace package (using the above rules), then “foo” is immediately created as a package. The creation of the namespace package is not deferred until a sub-level import occurs.

A namespace package is not fundamentally different from a regular package. It is just a different way of creating packages. Once a namespace package is created, there is no functional difference between it and a regular package.

Dynamic path computation

The import machinery will behave as if a namespace package’s __path__ is recomputed before each portion is loaded.

For performance reasons, it is expected that this will be achieved by detecting that the parent path has changed. If no change has taken place, then no __path__ recomputation is required. The implementation must ensure that changes to the contents of the parent path are detected, as well as detecting the replacement of the parent path with a new path entry list object.

Impact on import finders and loaders

The specification expands PEP 302 loaders to include an optional method called module_repr() which if present, is used to generate module object reprs. See the section below for further details.

Differences between namespace packages and regular packages

Namespace packages and regular packages are very similar. The differences are:

Namespace packages in the standard library

It is possible, and this PEP explicitly allows, that parts of the standard library be implemented as namespace packages. When and if any standard library packages become namespace packages is outside the scope of this PEP.

Migrating from legacy namespace packages

As described above, prior to this PEP pkgutil.extend_path() was used by legacy portions to create namespace packages. Because it is likely not practical for all existing portions of a namespace package to be migrated to this PEP at once, extend_path() will be modified to also recognize PEP 420 namespace packages. This will allow some portions of a namespace to be legacy portions while others are migrated to PEP 420. These hybrid namespace packages will not have the dynamic path computation that normal namespace packages have, since extend_path() never provided this functionality in the past.

Packaging Implications

Multiple portions of a namespace package can be installed into the same directory, or into separate directories. For this section, suppose there are two portions which define “foo.bar” and “foo.baz”. “foo” itself is a namespace package.

Note that “foo.bar” and “foo.baz” can be installed into the same “foo” directory because they will not have any files in common.

Examples

Nested namespace packages

This example uses the following directory structure:

Here, both parent and child are namespace packages: Portions of them exist in different directories, and they do not have __init__.py files.

Dynamic path computation

This example uses a similar directory structure, but adds a third portion:

Discussion

At PyCon 2012, we had a discussion about namespace packages at which PEP 382 and PEP 402 were rejected, to be replaced by this PEP [3].

There is no intention to remove support of regular packages. If a developer knows that her package will never be a portion of a namespace package, then there is a performance advantage to it being a regular package (with an __init__.py ). Creation and loading of a regular package can take place immediately when it is located along the path. With namespace packages, all entries in the path must be scanned before the package is created.

Note that an ImportWarning will no longer be raised for a directory lacking an __init__.py file. Such a directory will now be imported as a namespace package, whereas in prior Python versions an ImportWarning would be raised.

Nick Coghlan presented a list of his objections to this proposal [4]. They are:

Nick later gave a detailed response to his own objections [5], which is summarized here:

find_module versus find_loader

An early draft of this PEP specified a change to the find_module method in order to support namespace packages. It would be modified to return a string in the case where a namespace package portion was discovered.

The use case for supporting multiple portions per find_loader call is given in [7].

Dynamic path computation

Guido raised a concern that automatic dynamic path computation was an unnecessary feature [8]. Later in that thread, PJ Eby and Nick Coghlan presented arguments as to why dynamic computation would minimize surprise to Python users. The conclusion of that discussion has been included in this PEP’s Rationale section.

An earlier version of this PEP required that dynamic path computation could only take affect if the parent path object were modified in-place. That is, this would work:

But this would not:

Module reprs

Previously, module reprs were hard coded based on assumptions about a module’s __file__ attribute. If this attribute existed and was a string, it was assumed to be a file system path, and the module object’s repr would include this in its value. The only exception was that PEP 302 reserved missing __file__ attributes to built-in modules, and in CPython, this assumption was baked into the module object’s implementation. Because of this restriction, some modules contained contrived __file__ values that did not reflect file system paths, and which could cause unexpected problems later (e.g. os.path.join() on a non-path __file__ would return gibberish).

This PEP relaxes this constraint, and leaves the setting of __file__ to the purview of the loader producing the module. Loaders may opt to leave __file__ unset if no file system path is appropriate. Loaders may also set additional reserved attributes on the module if useful. This means that the definitive way to determine the origin of a module is to check its __loader__ attribute.

For example, namespace packages as described in this PEP will have no __file__ attribute because no corresponding file exists. In order to provide flexibility and descriptiveness in the reprs of such modules, a new optional protocol is added to PEP 302 loaders. Loaders can implement a module_repr() method which takes a single argument, the module object. This method should return the string to be used verbatim as the repr of the module. The rules for producing a module repr are now standardized as:

Here is a snippet showing how namespace module reprs are calculated from its loader:

Built-in module reprs would no longer need to be hard-coded, but instead would come from their loader as well:

Here are some example reprs of different types of modules with different sets of the related attributes:

References

Copyright

This document has been placed in the public domain.

Understand Python Namespace and Scope with Examples

In this class, we’ll cover what is Python namespace and why is it needed? We’ll also talk about what is scope in Python and how namespaces can be used to implement it.

The concept of namespaces is not limited to any particular programming language. C/C++ and Java also have it where it works as a means to distinguish between different sections of a program.

The body of a section may consist of a method, or a function, or all the methods of a class. So, a namespace is a practical approach to define the scope, and it helps to avoid name conflicts.

While in Python, the namespace is a fundamental idea to structure and organize the code, especially more useful in large projects. However, it could be a bit difficult concept to grasp if you’re new to programming. Hence, we tried to make namespaces just a little easier to understand.

Python Namespace and Scope

What are names in Python?

Before getting on to namespaces, first, let’s understand what Python means by a name.

A name in Python is just a way to access a variable like in any other languages. However, Python is more flexible when it comes to the variable declaration. You can declare a variable by just assigning a name to it.

You can use names to reference values.

You can even assign a name to a function.

You can also assign a name and then reuse it. Check the below example; it is alright for a name to point to different values.

And here is the output follows.

So, you can see that one name is working perfectly fine to hold data of different types.

You can learn more about types in Python from here – Python data types.

The naming mechanism works inline with Python’s object system, i.e., everything in Python is an object. All the data types such as numbers, strings, functions, classes are all objects. And a name acts as a reference to get to the objects.

What are namespaces in Python?

A namespace is a simple system to control the names in a program. It ensures that names are unique and won’t lead to any conflict.

Also, add to your knowledge that Python implements namespaces in the form of dictionaries. It maintains a name-to-object mapping where names act as keys and the objects as values. Multiple namespaces may have the same name but pointing to a different variable. Check out a few examples of namespaces for more clarity.

Local Namespace

This namespace covers the local names inside a function. Python creates this namespace for every function called in a program. It remains active until the function returns.

Global Namespace

This namespace covers the names from various imported modules used in a project. Python creates this namespace for every module included in your program. It’ll last until the program ends.

Built-in Namespace

This namespace covers the built-in functions and built-in exception names. Python creates it as the interpreter starts and keeps it until you exit.

What is Scope in Python?

Namespaces make our programs immune from name conflicts. However, it doesn’t give us a free ride to use a variable name anywhere we want. Python restricts names to be bound by specific rules known as a scope. The scope determines the parts of the program where you could use that name without any prefix.

Scope Resolution in Python – Examples

Scope resolution for a given name begins from the inner-most function and then goes higher and higher until the program finds the related object. If the search ends without any outcome, then the program throws a NameError exception.

Let’s now see some examples which you can run inside any Python IDE or with IDLE.

The output is as follows.

In this example, we used the dir() function. It lists all the names that are available in a Python program then.

In the first print() statement, the dir() only displays the list of names inside the current scope. While in the second print(), it finds only one name, “b_var,” a local function variable.

Calling dir() after defining the foo() pushes it to the list of names available in the global namespace.

In the next example, we’ll see the list of names inside some nested functions. The code in this block continues from the previous block.

The output is as follows.

The above example defines two variables and a function inside the scope of outer_foo(). Inside the inner_foo(), the dir() function only displays one name i.e. “inner_var”. It is alright as the “inner_var” is the only variable defined in there.

If you reuse a global name inside a local namespace, then Python creates a new local variable with the same name.

Here goes the output of the above code after execution.

We’ve declared a global variable as “a_var” inside both the outer_foo() and inner_foo() functions. However, we’ve assigned different values in the same global variable. And that’s the reason the value of “a_var” is same (i.e., 4) on all occasions.

Whereas, each function is creating its own “b_var” variable inside the local scope. And the print() function is showing the values of this variable as per its local context.

How to correctly import modules in Python?

It is very likely that you would import some of the external modules in your program. So, we’ll discuss here some of the import strategies, and you can choose the best one.

Import all names from a module

It’ll import all the names from a module directly into your working namespace. Since it is an effortless way, so you might tempt to use this method. However, you may not be able to tell which module imported a particular function.

Here is an example of using this method.

The output of the above code is as follows.

In this example, we’ve imported two distinct math modules, one after the other. There are some common names which both of these modules have. So, the second module will override the definitions of functions in the first.

The first call to sqrt() returns a real number and the second one gives a complex number. And now, there is no way we can call the sqrt() function from the first math module.

Even if we call the function using the module name, then Python will raise the NameError exception. So, the lesson learned here is that there are no shortcuts for quality code.

Import specific names from a module

If you are sure of the names to be used from a module, then import them directly into your program. This method is slightly better but will not prevent you from polluting the namespace completely. It is because you can’t use any other name from the module. Here also, any function having the same name in your program will also override the same definition present in the module. The affected method will become dormant in such a case.

Check out an example of using this method.

The output of the above code is as follows.

Import just the module using its name

It is the most reliable and suggested way of importing a module. However, it comes with a catch that you need to prefix the name of the module before using any name from it. But you can prevent the program from polluting the namespace and freely define functions with matching names in the module.

The output of the above example goes like this.

Quick wrap up – Python Namespace and Scope

If you want to do serious programming, then it is vital for you to know how scopes and namespaces work in Python. With this knowledge, you can even develop a scalable package ecosystem to be used by a large group working on a massive project.

If you find something new to learn today, then do share it with others. And, follow us on our social media (Facebook/Twitter) accounts to see more of this.