pyspark.sql.functions.desc#

pyspark.sql.functions.desc(col)[source]#

Returns a sort expression for the target column in descending order. This function is used in sort and orderBy functions.

New in version 1.3.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colColumn or str: Target column to sort by in the descending order.

Returns

Column: The column specifying the sort order.

Examples

Example 1: Sort DataFrame by ‘id’ column in descending order.

>>> from pyspark.sql.functions import desc
>>> df = spark.createDataFrame([(4, 'B'), (3, 'A'), (2, 'C')], ['id', 'value'])
>>> df.sort(desc("id")).show()
+---+-----+
| id|value|
+---+-----+
|  4|    B|
|  3|    A|
|  2|    C|
+---+-----+

Example 2: Use desc in orderBy function to sort the DataFrame.

>>> from pyspark.sql.functions import desc
>>> df = spark.createDataFrame([(4, 'B'), (3, 'A'), (2, 'C')], ['id', 'value'])
>>> df.orderBy(desc("value")).show()
+---+-----+
| id|value|
+---+-----+
|  2|    C|
|  4|    B|
|  3|    A|
+---+-----+

Example 3: Combine asc with desc to sort by multiple columns.

>>> from pyspark.sql.functions import asc, desc
>>> df = spark.createDataFrame([(2, 'A', 4),
...                             (1, 'B', 3),
...                             (3, 'A', 2)], ['id', 'group', 'value'])
>>> df.sort(desc("group"), asc("value")).show()
+---+-----+-----+
| id|group|value|
+---+-----+-----+
|  1|    B|    3|
|  3|    A|    2|
|  2|    A|    4|
+---+-----+-----+

Example 4: Implement desc from column expression.

>>> df = spark.createDataFrame([(4, 'B'), (3, 'A'), (2, 'C')], ['id', 'value'])
>>> df.sort(df.id.desc()).show()
+---+-----+
| id|value|
+---+-----+
|  4|    B|
|  3|    A|
|  2|    C|
+---+-----+