11.1.2 Data Types
The grammar rules for expressions that apply procedures are:
In Python, nearly every data value (including lists and strings) is an object. This means the way we manipulate data is to invoke methods on objects. To invoke a method we use the same rules, but the PrimaryExpression of the CallExpression specifies an object and method:
The name AttributeReference is used since the same syntax is used for accessing the internal state of objects as well.
tokenize procedure includes five method applications, four of which are
tokens.append(current). The object reference is
tokens, the list of tokens in the input. The list
append method takes one parameter and adds that value to the end of the list.
The other method invocation is
c is a string consisting of one character in the input. The
isspace method for the string datatype returns true if the input string is non-empty and all characters in the string are whitespace (either spaces, tabs, or newlines).
tokenize procedure also uses the built-in function
len which takes as input an object of a collection datatype such as a list or a string, and outputs the number of elements in the collection. It is a procedure, not a method; the input object is passed in as a parameter. In
tokenize, we use
len(current) to find the number of characters in the current token.