```
The `filter` method takes a function like `search` but returns a `Selectors` instance of all matching [Selector](#selector) instances:
```python
# Find all products with prices over $50
>>> filtering_function = lambda p: float(p.css('.price_color').re_first(r'[\d\.]+')) > 50
>>> page.css('.product_pod').filter(filtering_function)
[
,
,
,
...]
```
Safe access to the first or last element without index errors:
```python
>>> page.css('.product').first # First Selector or None
>>> page.css('.product').last # Last Selector or None
>>> page.css('.nonexistent').first # Returns None instead of raising IndexError
```
Get the number of [Selector](#selector) instances in a [Selectors](#selectors) instance:
```python
page.css('.product_pod').length
```
which is equivalent to
```python
len(page.css('.product_pod'))
```
## TextHandler
All methods/properties that return a string return `TextHandler`, and those that return a list of strings return [TextHandlers](#texthandlers) instead.
TextHandler is a subclass of the standard Python string, so all standard string operations are supported.
TextHandler provides extra methods and properties beyond standard Python strings. All methods and properties in all classes that return string(s) return TextHandler, enabling chaining and cleaner code. It can also be imported directly and used on any string.
### Usage
All operations (slicing, indexing, etc.) and methods (`split`, `replace`, `strip`, etc.) return a `TextHandler`, so they can be chained.
The `re` and `re_first` methods exist in [Selector](#selector), [Selectors](#selectors), and [TextHandlers](#texthandlers) as well, accepting the same arguments.
- The `re` method takes a string/compiled regex pattern as the first argument. It searches the data for all strings matching the regex and returns them as a [TextHandlers](#texthandlers) instance. The `re_first` method takes the same arguments but returns only the first result as a `TextHandler` instance.
Also, it takes other helpful arguments, which are:
- **replace_entities**: This is enabled by default. It replaces character entity references with their corresponding characters.
- **clean_match**: It's disabled by default. This causes the method to ignore all whitespace, including consecutive spaces, while matching.
- **case_sensitive**: It's enabled by default. As the name implies, disabling it causes the regex to ignore letter case during compilation.
The return result is [TextHandlers](#texthandlers) because the `re` method is used:
```python
>>> page.css('.price_color').re(r'[\d\.]+')
['51.77',
'53.74',
'50.10',
'47.82',
'54.23',
...]
>>> page.css('.product_pod h3 a::attr(href)').re(r'catalogue/(.*)/index.html')
['a-light-in-the-attic_1000',
'tipping-the-velvet_999',
'soumission_998',
'sharp-objects_997',
...]
```
Examples with custom strings demonstrating the other arguments:
```python
>>> from scrapling import TextHandler
>>> test_string = TextHandler('hi there') # Hence the two spaces
>>> test_string.re('hi there')
>>> test_string.re('hi there', clean_match=True) # Using `clean_match` will clean the string before matching the regex
['hi there']
>>> test_string2 = TextHandler('Oh, Hi Mark')
>>> test_string2.re_first('oh, hi Mark')
>>> test_string2.re_first('oh, hi Mark', case_sensitive=False) # Hence disabling `case_sensitive`
'Oh, Hi Mark'
# Mixing arguments
>>> test_string.re('hi there', clean_match=True, case_sensitive=False)
['hi There']
```
Since `html_content` returns `TextHandler`, regex can be applied directly on HTML content:
```python
>>> page.html_content.re('div class=".*">(.*)>> page.css('#page-data::text').get()
'\n {\n "lastUpdated": "2024-09-22T10:30:00Z",\n "totalProducts": 3\n }\n '
>>> page.css('#page-data::text').get().json()
{'lastUpdated': '2024-09-22T10:30:00Z', 'totalProducts': 3}
```
If no text node is specified while selecting an element, the text content is selected automatically:
```python
>>> page.css('#page-data')[0].json()
{'lastUpdated': '2024-09-22T10:30:00Z', 'totalProducts': 3}
```
The [Selector](#selector) class adds additional behavior. Given this page:
```html
```
The [Selector](#selector) class has the `get_all_text` method, which returns a `TextHandler`. For example:
```python
>>> page.css('div::text').get().json()
```
This throws an error because the `div` tag has no direct text content. The `get_all_text` method handles this case:
```python
>>> page.css('div')[0].get_all_text(ignore_tags=[]).json()
{'lastUpdated': '2024-09-22T10:30:00Z', 'totalProducts': 3}
```
The `ignore_tags` argument is used here because its default value is `('script', 'style',)`.
When dealing with a JSON response:
```python
>>> page = Selector("""{"some_key": "some_value"}""")
```
The [Selector](#selector) class is optimized for HTML, so it treats this as a broken HTML response and wraps it. The `html_content` property shows:
```python
>>> page.html_content
'{"some_key": "some_value"}
'
```
The `json` method can be used directly:
```python
>>> page.json()
{'some_key': 'some_value'}
```
For JSON responses, the [Selector](#selector) class keeps a raw copy of the content it receives. When `.json()` is called, it checks for that raw copy first and converts it to JSON. If the raw copy is unavailable (as with sub-elements), it checks the current element's text content, then falls back to `get_all_text`.
- The `.clean()` method removes all whitespace and consecutive spaces, returning a new `TextHandler` instance:
```python
>>> TextHandler('\n wonderful idea, \reh?').clean()
'wonderful idea, eh?'
```
The `remove_entities` argument causes `clean` to replace HTML entities with their corresponding characters.
- The `.sort()` method sorts the string characters:
```python
>>> TextHandler('acb').sort()
'abc'
```
Or do it in reverse:
```python
>>> TextHandler('acb').sort(reverse=True)
'cba'
```
This class is returned in place of strings nearly everywhere in the library.
## TextHandlers
This class inherits from standard lists, adding `re` and `re_first` as new methods.
The `re_first` method runs `re` on each [TextHandler](#texthandler) and returns the first result, or `None`.
## AttributesHandler
This is a read-only version of Python's standard dictionary, or `dict`, used solely to store the attributes of each element/[Selector](#selector) instance.
```python
>>> print(page.find('script').attrib)
{'id': 'page-data', 'type': 'application/json'}
>>> type(page.find('script').attrib).__name__
'AttributesHandler'
```
Because it's read-only, it will use fewer resources than the standard dictionary. Still, it has the same dictionary method and properties, except those that allow you to modify/override the data.
It currently adds two extra simple methods:
- The `search_values` method
Searches the current attributes by values (rather than keys) and returns a dictionary of each matching item.
A simple example would be
```python
>>> for i in page.find('script').attrib.search_values('page-data'):
print(i)
{'id': 'page-data'}
```
But this method provides the `partial` argument as well, which allows you to search by part of the value:
```python
>>> for i in page.find('script').attrib.search_values('page', partial=True):
print(i)
{'id': 'page-data'}
```
A more practical example is using it with `find_all` to find all elements that have a specific value in their attributes:
```python
>>> page.find_all(lambda element: list(element.attrib.search_values('product')))
[,
,
]
```
All these elements have 'product' as the value for the `class` attribute.
The `list` function is used here because `search_values` returns a generator, so it would be `True` for all elements.
- The `json_string` property
This property converts current attributes to a JSON string if the attributes are JSON serializable; otherwise, it throws an error.
```python
>>>page.find('script').attrib.json_string
b'{"id":"page-data","type":"application/json"}'
```