| Lists: | pgsql-docspgsql-hackers |
|---|
| From: | Vik Fearing <vik(at)postgresfriends(dot)org> |
|---|---|
| To: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | GROUP BY DISTINCT |
| Date: | 2021-02-21 12:52:24 |
| Message-ID: | bf3805a8-d7d1-ae61-fece-761b7ff41ecc@postgresfriends.org |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
When combining multiple grouping items, such as rollups and cubes, the
resulting flattened grouping sets can contain duplicate items. The
standard provides for this by allowing GROUP BY DISTINCT to deduplicate
them prior to doing the actual work.
For example:
GROUP BY ROLLUP (a,b), ROLLUP (a,c)
expands to the sets:
(a,b,c), (a,b), (a,b), (a,c), (a), (a), (a,c), (a), ()
but:
GROUP BY DISTINCT ROLLUP (a,b), ROLLUP (a,c)
expands to just the sets:
(a,b,c), (a,b), (a,c), (a), ()
Attached is a patch to implement this for PostgreSQL.
--
Vik Fearing
| Attachment | Content-Type | Size |
|---|---|---|
| 0001-implement-GROUP-BY-DISTINCT.v01.patch | text/x-patch | 19.8 KB |
| From: | er(at)xs4all(dot)nl |
|---|---|
| To: | Vik Fearing <vik(at)postgresfriends(dot)org>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-02-21 14:06:03 |
| Message-ID: | 2056725465.229405.1613916363889@webmailclassic.xs4all.nl |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
> On 2021.02.21. 13:52 Vik Fearing <vik(at)postgresfriends(dot)org> wrote:
>
> Attached is a patch to implement this for PostgreSQL.
> []
The changed line that gets stuffed into sql_features is missing a terminal value (to fill the 'comments' column).
This line:
'+T434 GROUP BY DISTINCT YES'
(A tab at the end will do, I suppose; that's how I fixed the patch locally)
Erik Rijkers
| From: | Vik Fearing <vik(at)postgresfriends(dot)org> |
|---|---|
| To: | er(at)xs4all(dot)nl, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-02-21 14:14:12 |
| Message-ID: | c350ff6b-d70d-9952-be23-a18855b62589@postgresfriends.org |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
On 2/21/21 3:06 PM, er(at)xs4all(dot)nl wrote:
>> On 2021.02.21. 13:52 Vik Fearing <vik(at)postgresfriends(dot)org> wrote:
>>
>> Attached is a patch to implement this for PostgreSQL.
>> []
>
> The changed line that gets stuffed into sql_features is missing a terminal value (to fill the 'comments' column).
> This line:
> '+T434 GROUP BY DISTINCT YES'
>
> (A tab at the end will do, I suppose; that's how I fixed the patch locally)
Argh. Fixed.
Thank you for looking at it!
--
Vik Fearing
| Attachment | Content-Type | Size |
|---|---|---|
| 0001-implement-GROUP-BY-DISTINCT.v02.patch | text/x-patch | 19.8 KB |
| From: | Georgios Kokolatos <gkokolatos(at)protonmail(dot)com> |
|---|---|
| To: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Cc: | Vik Fearing <vik(at)postgresfriends(dot)org> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-02 15:06:28 |
| Message-ID: | 161469758801.29967.4455067072614569957.pgcf@coridan.postgresql.org |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
The following review has been posted through the commitfest application:
make installcheck-world: not tested
Implements feature: not tested
Spec compliant: not tested
Documentation: not tested
Hi,
this is a useful feature, thank you for implementing. I gather that it follows the standard, if so,
then there are definitely no objections from me.
The patch in version 2, applies cleanly and passes all the tests.
It contains documentation which seems correct to a non native speaker.
As a minor gripe, I would note the addition of list_int_cmp.
The block
+ /* Sort each groupset individually */
+ foreach(cell, result)
+ list_sort(lfirst(cell), list_int_cmp);
Can follow suit from the rest of the code, and define a static cmp_list_int_asc(), as
indeed the same patch does for cmp_list_len_contents_asc.
This is indeed point of which I will not hold a too strong opinion.
Overall :+1: from me.
I will be bumping to 'Ready for Committer' unless objections.
| From: | Vik Fearing <vik(at)postgresfriends(dot)org> |
|---|---|
| To: | Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Cc: | Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-02 16:51:52 |
| Message-ID: | 4294e82f-a134-5dbb-d9d6-03d72dce2e1e@postgresfriends.org |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
On 3/2/21 4:06 PM, Georgios Kokolatos wrote:
> As a minor gripe, I would note the addition of list_int_cmp.
> The block
>
> + /* Sort each groupset individually */
> + foreach(cell, result)
> + list_sort(lfirst(cell), list_int_cmp);
>
> Can follow suit from the rest of the code, and define a static cmp_list_int_asc(), as
> indeed the same patch does for cmp_list_len_contents_asc.
> This is indeed point of which I will not hold a too strong opinion.
I did it this way because list_int_cmp is a general purpose function for
int lists that can be reused elsewhere in the future. Whereas
cmp_list_len_contents_asc is very specific to this case so I kept it local.
I'm happy to change it around if that's what consensus wants.
> Overall :+1: from me.
Thanks for looking at it!
> I will be bumping to 'Ready for Committer' unless objections.
In that case, here is another patch that fixes a typo in the docs
mentioned privately to me by Erik. The typo (and a gratuitous rebase)
is the only change to what you just reviewed.
--
Vik Fearing
| Attachment | Content-Type | Size |
|---|---|---|
| 0001-implement-GROUP-BY-DISTINCT.v03.patch | text/x-patch | 19.8 KB |
| From: | Georgios <gkokolatos(at)protonmail(dot)com> |
|---|---|
| To: | Vik Fearing <vik(at)postgresfriends(dot)org> |
| Cc: | "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-02 19:21:42 |
| Message-ID: | AxJsCzbFuyPfQckgXQ-HOMFvSqwMFszDbEJglCEe0Nijjn8tjNoNvbTGgjlfmZoPF7zhdW_ZwIdMvL1N8x44RTYnmpe3GzGRDE4tGIymuec=@protonmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, March 2, 2021 5:51 PM, Vik Fearing <vik(at)postgresfriends(dot)org> wrote:
> On 3/2/21 4:06 PM, Georgios Kokolatos wrote:
>
> > As a minor gripe, I would note the addition of list_int_cmp.
> > The block
> >
> > - /* Sort each groupset individually */
> >
> >
> > - foreach(cell, result)
> >
> >
> > - list_sort(lfirst(cell), list_int_cmp);
> >
> >
> >
> > Can follow suit from the rest of the code, and define a static cmp_list_int_asc(), as
> > indeed the same patch does for cmp_list_len_contents_asc.
> > This is indeed point of which I will not hold a too strong opinion.
>
> I did it this way because list_int_cmp is a general purpose function for
> int lists that can be reused elsewhere in the future. Whereas
> cmp_list_len_contents_asc is very specific to this case so I kept it local.
Of course. I got the intention and I have noted my opinion.
>
> I'm happy to change it around if that's what consensus wants.
As before, I will not hold a too strong opinion.
>
> > Overall :+1: from me.
>
> Thanks for looking at it!
>
> > I will be bumping to 'Ready for Committer' unless objections.
>
> In that case, here is another patch that fixes a typo in the docs
> mentioned privately to me by Erik. The typo (and a gratuitous rebase)
> is the only change to what you just reviewed.
Thank you. The typo was indistiguishable to me too.
My :+1: stands for version 3 of the patch. Updating status in the
commitfest to reflect that.
//Georgios -- https://www.vmware.com
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Vik Fearing
| From: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
|---|---|
| To: | Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Cc: | Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-12 23:33:36 |
| Message-ID: | b95ee556-c527-04b8-6cae-f55950dde94b@enterprisedb.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
Hi Vik,
The patch seems quite ready, I have just two comments.
1) Shouldn't this add another <indexterm> for DISTINCT, somewhere in the
documentation? Now the index points just to the SELECT DISTINCT part.
2) The part in gram.y that wraps/unwraps the boolean flag as an integer,
in order to stash it in the group lists is rather ugly, IMHO. It forces
all the places handling the list to be aware of this (there are not
many, but still ...). And there are no other places doing (bool) intVal
so it's not like there's a precedent for this.
I think the clean solution is to make group_clause produce a struct with
two fields, and just use that. Not sure how invasive that will be
outside gram.y, though.
Also, the all_or_distinct vs. distinct_or_all seems a bit error-prone. I
wonder if we can come up with some clearer names, describing the context
of those types.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
| From: | Pantelis Theodosiou <ypercube(at)gmail(dot)com> |
|---|---|
| To: | pgsql-docs(at)lists(dot)postgresql(dot)org |
| Subject: | Fwd: GROUP BY DISTINCT |
| Date: | 2021-03-13 01:03:19 |
| Message-ID: | CAE3TBxyAKwqJodfbOCdWy2ome1=ojyaboqiqTmdYCBG_1=JCZw@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
---------- Forwarded message ---------
From: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Date: Fri, Mar 12, 2021 at 11:33 PM
Subject: Re: GROUP BY DISTINCT
To: Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <
gkokolatos(at)protonmail(dot)com>, <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Erik Rijkers <er(at)xs4all(dot)nl>
Hi Vik,
The patch seems quite ready, I have just two comments.
1) Shouldn't this add another <indexterm> for DISTINCT, somewhere in the
documentation? Now the index points just to the SELECT DISTINCT part.
.....
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
After reading the above thread in hackers, I noticed that the index does
not point to aggrgeate functions either and DISTINCT is not mentioned in
the aggregate functions page either:
https://www.postgresql.org/docs/current/functions-aggregate.html
Shouldn't it be mentioned with an example of COUNT(DISTINCT ...) or
aggregate_function(DISTINCT ...) in general ?
Best regards
Pantelis Theodosiou
| From: | Vik Fearing <vik(at)postgresfriends(dot)org> |
|---|---|
| To: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Cc: | Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-16 08:21:03 |
| Message-ID: | 3817d5ec-f3e4-e4b9-d169-271ae5d3cbe4@postgresfriends.org |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
On 3/13/21 12:33 AM, Tomas Vondra wrote:
> Hi Vik,
>
> The patch seems quite ready, I have just two comments.
Thanks for taking a look.
> 1) Shouldn't this add another <indexterm> for DISTINCT, somewhere in the
> documentation? Now the index points just to the SELECT DISTINCT part.
Good idea; I never think about the index.
> 2) The part in gram.y that wraps/unwraps the boolean flag as an integer,
> in order to stash it in the group lists is rather ugly, IMHO. It forces
> all the places handling the list to be aware of this (there are not
> many, but still ...). And there are no other places doing (bool) intVal
> so it's not like there's a precedent for this.
There is kind of a precedent for it, I was copying off of TriggerEvents
and func_alias_clause.
> I think the clean solution is to make group_clause produce a struct with
> two fields, and just use that. Not sure how invasive that will be
> outside gram.y, though.
I didn't want to create a whole new parse node for it, but Andrew Gierth
pointed me towards SelectLimit so I did it like that and I agree it is
much cleaner.
> Also, the all_or_distinct vs. distinct_or_all seems a bit error-prone. I
> wonder if we can come up with some clearer names, describing the context
> of those types.
I turned this into an enum for ALL/DISTINCT/default and the caller can
choose what it wants to do with default. I think that's a lot cleaner,
too. Maybe DISTINCT ON should be changed to fit in that? I left it
alone for now.
I also snuck in something that all of us overlooked which is outputting
the DISTINCT in ruleutils.c. I didn't add a test for it but that would
have been an unfortunate bug.
New patch attached, rebased on 15639d5e8f.
--
Vik Fearing
| Attachment | Content-Type | Size |
|---|---|---|
| 0001-implement-GROUP-BY-DISTINCT.v04.patch | text/x-patch | 23.9 KB |
| From: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
|---|---|
| To: | Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Cc: | Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-16 14:52:52 |
| Message-ID: | 8c44df08-bb64-3d78-da1a-d24b7c9db39d@enterprisedb.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
On 3/16/21 9:21 AM, Vik Fearing wrote:
> On 3/13/21 12:33 AM, Tomas Vondra wrote:
>> Hi Vik,
>>
>> The patch seems quite ready, I have just two comments.
>
> Thanks for taking a look.
>
>> 1) Shouldn't this add another <indexterm> for DISTINCT, somewhere in the
>> documentation? Now the index points just to the SELECT DISTINCT part.
>
> Good idea; I never think about the index.
>
>> 2) The part in gram.y that wraps/unwraps the boolean flag as an integer,
>> in order to stash it in the group lists is rather ugly, IMHO. It forces
>> all the places handling the list to be aware of this (there are not
>> many, but still ...). And there are no other places doing (bool) intVal
>> so it's not like there's a precedent for this.
>
> There is kind of a precedent for it, I was copying off of TriggerEvents
> and func_alias_clause.
>
I see. I was looking for "(bool) intVal" but you're right TriggerEvents
code does something similar.
>> I think the clean solution is to make group_clause produce a struct with
>> two fields, and just use that. Not sure how invasive that will be
>> outside gram.y, though.
>
> I didn't want to create a whole new parse node for it, but Andrew Gierth
> pointed me towards SelectLimit so I did it like that and I agree it is
> much cleaner.
>
I agree, that's much cleaner.
>> Also, the all_or_distinct vs. distinct_or_all seems a bit error-prone. I
>> wonder if we can come up with some clearer names, describing the context
>> of those types.
>
> I turned this into an enum for ALL/DISTINCT/default and the caller can
> choose what it wants to do with default. I think that's a lot cleaner,
> too. Maybe DISTINCT ON should be changed to fit in that? I left it
> alone for now.
>
I think DISTINCT ON is a different kind of animal, because that is a
list of expressions, not just a simple enum state.
> I also snuck in something that all of us overlooked which is outputting
> the DISTINCT in ruleutils.c. I didn't add a test for it but that would
> have been an unfortunate bug.
>
Oh!
> New patch attached, rebased on 15639d5e8f.
>
Thanks. At this point it seems fine to me, no further comments.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
| From: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
|---|---|
| To: | Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Cc: | Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-18 17:25:47 |
| Message-ID: | 7137947a-87be-d04a-bf91-f42e47520daa@enterprisedb.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
On 3/16/21 3:52 PM, Tomas Vondra wrote:
>
>
> On 3/16/21 9:21 AM, Vik Fearing wrote:
>> On 3/13/21 12:33 AM, Tomas Vondra wrote:
>>> Hi Vik,
>>>
>>> The patch seems quite ready, I have just two comments.
>>
>> Thanks for taking a look.
>>
>>> 1) Shouldn't this add another <indexterm> for DISTINCT, somewhere in the
>>> documentation? Now the index points just to the SELECT DISTINCT part.
>>
>> Good idea; I never think about the index.
>>
>>> 2) The part in gram.y that wraps/unwraps the boolean flag as an integer,
>>> in order to stash it in the group lists is rather ugly, IMHO. It forces
>>> all the places handling the list to be aware of this (there are not
>>> many, but still ...). And there are no other places doing (bool) intVal
>>> so it's not like there's a precedent for this.
>>
>> There is kind of a precedent for it, I was copying off of TriggerEvents
>> and func_alias_clause.
>>
>
> I see. I was looking for "(bool) intVal" but you're right TriggerEvents
> code does something similar.
>
>>> I think the clean solution is to make group_clause produce a struct with
>>> two fields, and just use that. Not sure how invasive that will be
>>> outside gram.y, though.
>>
>> I didn't want to create a whole new parse node for it, but Andrew Gierth
>> pointed me towards SelectLimit so I did it like that and I agree it is
>> much cleaner.
>>
>
> I agree, that's much cleaner.
>
>>> Also, the all_or_distinct vs. distinct_or_all seems a bit error-prone. I
>>> wonder if we can come up with some clearer names, describing the context
>>> of those types.
>>
>> I turned this into an enum for ALL/DISTINCT/default and the caller can
>> choose what it wants to do with default. I think that's a lot cleaner,
>> too. Maybe DISTINCT ON should be changed to fit in that? I left it
>> alone for now.
>>
>
> I think DISTINCT ON is a different kind of animal, because that is a
> list of expressions, not just a simple enum state.
>
>> I also snuck in something that all of us overlooked which is outputting
>> the DISTINCT in ruleutils.c. I didn't add a test for it but that would
>> have been an unfortunate bug.
>>
>
> Oh!
>
>> New patch attached, rebased on 15639d5e8f.
>>
>
> Thanks. At this point it seems fine to me, no further comments.
>
Pushed. Thanks for the patch.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
| From: | Pantelis Theodosiou <ypercube(at)gmail(dot)com> |
|---|---|
| To: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, pgsql-docs(at)lists(dot)postgresql(dot)org |
| Subject: | DISTINCT term in aggregate function |
| Date: | 2021-03-18 18:03:03 |
| Message-ID: | CAE3TBxyiC0ivwhAVAeL8C8G6UUpyQ+WKUrEnRZ7LsE-LRXzR8A@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
Hi, I didn't think of including you in this suggestion.
Or the pdsql-docs was not the right list to post? I didn't want to mix it
with the GROUP BY DISTINCT patch.
Please check my suggestion.
Best regards
Pantelis Theodosiou
---------- Forwarded message ---------
From: Pantelis Theodosiou <ypercube(at)gmail(dot)com>
Date: Sat, Mar 13, 2021 at 1:03 AM
Subject: Fwd: GROUP BY DISTINCT
To: <pgsql-docs(at)lists(dot)postgresql(dot)org>
---------- Forwarded message ---------
From: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Date: Fri, Mar 12, 2021 at 11:33 PM
Subject: Re: GROUP BY DISTINCT
To: Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <
gkokolatos(at)protonmail(dot)com>, <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Erik Rijkers <er(at)xs4all(dot)nl>
Hi Vik,
The patch seems quite ready, I have just two comments.
1) Shouldn't this add another <indexterm> for DISTINCT, somewhere in the
documentation? Now the index points just to the SELECT DISTINCT part.
.....
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
After reading the above thread in hackers, I noticed that the index does
not point to aggrgeate functions either and DISTINCT is not mentioned in
the aggregate functions page either:
https://www.postgresql.org/docs/current/functions-aggregate.html
Shouldn't it be mentioned with an example of COUNT(DISTINCT ...) or
aggregate_function(DISTINCT ...) in general ?
Best regards
Pantelis Theodosiou
| From: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
|---|---|
| To: | Pantelis Theodosiou <ypercube(at)gmail(dot)com>, pgsql-docs(at)lists(dot)postgresql(dot)org |
| Subject: | Re: DISTINCT term in aggregate function |
| Date: | 2021-03-18 18:05:52 |
| Message-ID: | 48e7af6c-a997-e361-3dec-1382a5824831@enterprisedb.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
Sorry, I'm not reading pgsql-docs very often, so I missed the post.
Yeah, we should probably add an indexterm to the other places too.
regards
On 3/18/21 7:03 PM, Pantelis Theodosiou wrote:
> Hi, I didn't think of including you in this suggestion.
> Or the pdsql-docs was not the right list to post? I didn't want to mix
> it with the GROUP BY DISTINCT patch.
>
> Please check my suggestion.
>
> Best regards
> Pantelis Theodosiou
>
>
>
>
> ---------- Forwarded message ---------
> From: *Pantelis Theodosiou* <ypercube(at)gmail(dot)com <mailto:ypercube(at)gmail(dot)com>>
> Date: Sat, Mar 13, 2021 at 1:03 AM
> Subject: Fwd: GROUP BY DISTINCT
> To: <pgsql-docs(at)lists(dot)postgresql(dot)org
> <mailto:pgsql-docs(at)lists(dot)postgresql(dot)org>>
>
>
>
> ---------- Forwarded message ---------
> From: *Tomas Vondra* <tomas(dot)vondra(at)enterprisedb(dot)com
> <mailto:tomas(dot)vondra(at)enterprisedb(dot)com>>
> Date: Fri, Mar 12, 2021 at 11:33 PM
> Subject: Re: GROUP BY DISTINCT
> To: Vik Fearing <vik(at)postgresfriends(dot)org
> <mailto:vik(at)postgresfriends(dot)org>>, Georgios Kokolatos
> <gkokolatos(at)protonmail(dot)com <mailto:gkokolatos(at)protonmail(dot)com>>,
> <pgsql-hackers(at)lists(dot)postgresql(dot)org
> <mailto:pgsql-hackers(at)lists(dot)postgresql(dot)org>>
> Cc: Erik Rijkers <er(at)xs4all(dot)nl <mailto:er(at)xs4all(dot)nl>>
>
>
> Hi Vik,
>
> The patch seems quite ready, I have just two comments.
>
> 1) Shouldn't this add another <indexterm> for DISTINCT, somewhere in the
> documentation? Now the index points just to the SELECT DISTINCT part.
>
> .....
>
> regards
>
> --
> Tomas Vondra
> EnterpriseDB: http://www.enterprisedb.com <http://www.enterprisedb.com>
> The Enterprise PostgreSQL Company
>
>
>
> After reading the above thread in hackers, I noticed that the index does
> not point to aggrgeate functions either and DISTINCT is not mentioned in
> the aggregate functions page
> either: https://www.postgresql.org/docs/current/functions-aggregate.html
> <https://www.postgresql.org/docs/current/functions-aggregate.html>
> Shouldn't it be mentioned with an example of COUNT(DISTINCT ...) or
> aggregate_function(DISTINCT ...) in general ?
>
> Best regards
>
> Pantelis Theodosiou
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
| From: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
|---|---|
| To: | Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Cc: | Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-18 19:27:40 |
| Message-ID: | 35077b31-2d62-1e31-0e2e-ddb52d590b73@enterprisedb.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
On 3/18/21 6:25 PM, Tomas Vondra wrote:
> On 3/16/21 3:52 PM, Tomas Vondra wrote:
>>
>>
>> On 3/16/21 9:21 AM, Vik Fearing wrote:
>>> On 3/13/21 12:33 AM, Tomas Vondra wrote:
>>>> Hi Vik,
>>>>
>>>> The patch seems quite ready, I have just two comments.
>>>
>>> Thanks for taking a look.
>>>
>>>> 1) Shouldn't this add another <indexterm> for DISTINCT, somewhere in the
>>>> documentation? Now the index points just to the SELECT DISTINCT part.
>>>
>>> Good idea; I never think about the index.
>>>
>>>> 2) The part in gram.y that wraps/unwraps the boolean flag as an integer,
>>>> in order to stash it in the group lists is rather ugly, IMHO. It forces
>>>> all the places handling the list to be aware of this (there are not
>>>> many, but still ...). And there are no other places doing (bool) intVal
>>>> so it's not like there's a precedent for this.
>>>
>>> There is kind of a precedent for it, I was copying off of TriggerEvents
>>> and func_alias_clause.
>>>
>>
>> I see. I was looking for "(bool) intVal" but you're right TriggerEvents
>> code does something similar.
>>
>>>> I think the clean solution is to make group_clause produce a struct with
>>>> two fields, and just use that. Not sure how invasive that will be
>>>> outside gram.y, though.
>>>
>>> I didn't want to create a whole new parse node for it, but Andrew Gierth
>>> pointed me towards SelectLimit so I did it like that and I agree it is
>>> much cleaner.
>>>
>>
>> I agree, that's much cleaner.
>>
>>>> Also, the all_or_distinct vs. distinct_or_all seems a bit error-prone. I
>>>> wonder if we can come up with some clearer names, describing the context
>>>> of those types.
>>>
>>> I turned this into an enum for ALL/DISTINCT/default and the caller can
>>> choose what it wants to do with default. I think that's a lot cleaner,
>>> too. Maybe DISTINCT ON should be changed to fit in that? I left it
>>> alone for now.
>>>
>>
>> I think DISTINCT ON is a different kind of animal, because that is a
>> list of expressions, not just a simple enum state.
>>
>>> I also snuck in something that all of us overlooked which is outputting
>>> the DISTINCT in ruleutils.c. I didn't add a test for it but that would
>>> have been an unfortunate bug.
>>>
>>
>> Oh!
>>
>>> New patch attached, rebased on 15639d5e8f.
>>>
>>
>> Thanks. At this point it seems fine to me, no further comments.
>>
>
> Pushed. Thanks for the patch.
>
Hmmm, this seems to fail on lapwing with this error:
parse_agg.c: In function 'expand_grouping_sets':
parse_agg.c:1851:23: error: value computed is not used
[-Werror=unused-value]
cc1: all warnings being treated as errors
That line is this:
foreach_delete_current(result, cell);
and I don't see how any of the values close by could be unused ...
The only possibility I can think of is some sort of issue in the old-ish
gcc release (4.7.2).
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
| From: | Thomas Munro <thomas(dot)munro(at)gmail(dot)com> |
|---|---|
| To: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
| Cc: | Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-18 21:02:53 |
| Message-ID: | CA+hUKG+rJkXWjj_WFPE8cDp+9Av2T3bQfEfEBd=W+PdWVCk8mg@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
On Fri, Mar 19, 2021 at 8:27 AM Tomas Vondra
<tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
> Hmmm, this seems to fail on lapwing with this error:
>
> parse_agg.c: In function 'expand_grouping_sets':
> parse_agg.c:1851:23: error: value computed is not used
> [-Werror=unused-value]
> cc1: all warnings being treated as errors
>
> That line is this:
>
> foreach_delete_current(result, cell);
>
> and I don't see how any of the values close by could be unused ...
>
> The only possibility I can think of is some sort of issue in the old-ish
> gcc release (4.7.2).
No sure what's going on there, but data points: I tried a 32 bit build
here (that's the other special thing about lapwing) and didn't see the
warning. GCC 10. Also curculio (gcc 4.2) and snapper (gcc 4.7) are
also showing this warning, but they don't have -Werror so they don't
fail. sidewinder (gcc 4.8) is not showing the warning.
| From: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
|---|---|
| To: | Thomas Munro <thomas(dot)munro(at)gmail(dot)com> |
| Cc: | Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-18 21:14:03 |
| Message-ID: | aa87b9b9-6a14-a84a-574f-e3ad2ce2504c@enterprisedb.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
On 3/18/21 10:02 PM, Thomas Munro wrote:
> On Fri, Mar 19, 2021 at 8:27 AM Tomas Vondra
> <tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
>> Hmmm, this seems to fail on lapwing with this error:
>>
>> parse_agg.c: In function 'expand_grouping_sets':
>> parse_agg.c:1851:23: error: value computed is not used
>> [-Werror=unused-value]
>> cc1: all warnings being treated as errors
>>
>> That line is this:
>>
>> foreach_delete_current(result, cell);
>>
>> and I don't see how any of the values close by could be unused ...
>>
>> The only possibility I can think of is some sort of issue in the old-ish
>> gcc release (4.7.2).
>
> No sure what's going on there, but data points: I tried a 32 bit build
> here (that's the other special thing about lapwing) and didn't see the
> warning. GCC 10. Also curculio (gcc 4.2) and snapper (gcc 4.7) are
> also showing this warning, but they don't have -Werror so they don't
> fail. sidewinder (gcc 4.8) is not showing the warning.
>
Thanks for the info. So it's likely related to older gcc releases. The
question is how to tweak the code to get rid of this ...
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
| From: | Thomas Munro <thomas(dot)munro(at)gmail(dot)com> |
|---|---|
| To: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
| Cc: | Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-18 21:57:13 |
| Message-ID: | CA+hUKG+6mHO82VCp9TYLcwf_x8qjd-jrxLnYCTpZOebmnrLzww@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
On Fri, Mar 19, 2021 at 10:14 AM Tomas Vondra
<tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
> >> The only possibility I can think of is some sort of issue in the old-ish
> >> gcc release (4.7.2).
> >
> > No sure what's going on there, but data points: I tried a 32 bit build
> > here (that's the other special thing about lapwing) and didn't see the
> > warning. GCC 10. Also curculio (gcc 4.2) and snapper (gcc 4.7) are
> > also showing this warning, but they don't have -Werror so they don't
> > fail. sidewinder (gcc 4.8) is not showing the warning.
> >
>
> Thanks for the info. So it's likely related to older gcc releases. The
> question is how to tweak the code to get rid of this ...
It's frustrating to have to do press-ups to fix a problem because a
zombie Debian 7 system is running with -Werror (though it's always
possible that it's telling us something interesting...). Anyway, I
think someone with a GCC < 4.8 compiler would have to investigate. I
was hoping to help, but none of my systems have one in easy-to-install
format...
| From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
|---|---|
| To: | Thomas Munro <thomas(dot)munro(at)gmail(dot)com> |
| Cc: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-18 22:35:42 |
| Message-ID: | 3818384.1616106942@sss.pgh.pa.us |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
Thomas Munro <thomas(dot)munro(at)gmail(dot)com> writes:
> On Fri, Mar 19, 2021 at 10:14 AM Tomas Vondra
> <tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
>> Thanks for the info. So it's likely related to older gcc releases. The
>> question is how to tweak the code to get rid of this ...
> It's frustrating to have to do press-ups to fix a problem because a
> zombie Debian 7 system is running with -Werror (though it's always
> possible that it's telling us something interesting...). Anyway, I
> think someone with a GCC < 4.8 compiler would have to investigate. I
> was hoping to help, but none of my systems have one in easy-to-install
> format...
Hmm ... prairiedog isn't showing the warning, but maybe gaur will.
I can take a look if nobody else is stepping up.
regards, tom lane
| From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
|---|---|
| To: | Thomas Munro <thomas(dot)munro(at)gmail(dot)com> |
| Cc: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-18 23:10:22 |
| Message-ID: | 3828009.1616109022@sss.pgh.pa.us |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
I wrote:
> Hmm ... prairiedog isn't showing the warning, but maybe gaur will.
Bingo:
parse_agg.c: In function 'expand_grouping_sets':
parse_agg.c:1851:5: warning: value computed is not used
This is gcc 4.5, but hopefully whatever shuts it up will also work on 4.7.
I'll work on figuring that out.
regards, tom lane
| From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
|---|---|
| To: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
| Cc: | Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-18 23:26:43 |
| Message-ID: | 3858396.1616110003@sss.pgh.pa.us |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
I wrote:
> This is gcc 4.5, but hopefully whatever shuts it up will also work on 4.7.
> I'll work on figuring that out.
Actually, the problem is pretty obvious after comparing this use
of foreach_delete_current() to every other one. I'm not sure why
the compiler warnings are phrased just as they are, but the fix
I just pushed does make 4.5 happy.
regards, tom lane
| From: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> |
|---|---|
| To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
| Cc: | Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Vik Fearing <vik(at)postgresfriends(dot)org>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-18 23:52:17 |
| Message-ID: | a0160421-654c-885e-1e90-5c21e9e1ab72@enterprisedb.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
On 3/19/21 12:26 AM, Tom Lane wrote:
> I wrote:
>> This is gcc 4.5, but hopefully whatever shuts it up will also work on 4.7.
>> I'll work on figuring that out.
>
> Actually, the problem is pretty obvious after comparing this use
> of foreach_delete_current() to every other one. I'm not sure why
> the compiler warnings are phrased just as they are, but the fix
> I just pushed does make 4.5 happy.
>
Thanks! Yeah, that looks obvious. Funny the older compilers noticed
this, not the new fancy ones.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
| From: | Vik Fearing <vik(at)postgresfriends(dot)org> |
|---|---|
| To: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
| Cc: | Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Georgios Kokolatos <gkokolatos(at)protonmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Erik Rijkers <er(at)xs4all(dot)nl> |
| Subject: | Re: GROUP BY DISTINCT |
| Date: | 2021-03-18 23:55:52 |
| Message-ID: | ea0cc900-174a-0f51-539f-5f51af8b4097@postgresfriends.org |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Lists: | pgsql-docs pgsql-hackers |
On 3/19/21 12:52 AM, Tomas Vondra wrote:
>
> On 3/19/21 12:26 AM, Tom Lane wrote:
>> I wrote:
>>> This is gcc 4.5, but hopefully whatever shuts it up will also work on 4.7.
>>> I'll work on figuring that out.
>>
>> Actually, the problem is pretty obvious after comparing this use
>> of foreach_delete_current() to every other one. I'm not sure why
>> the compiler warnings are phrased just as they are, but the fix
>> I just pushed does make 4.5 happy.
>>
>
> Thanks! Yeah, that looks obvious. Funny the older compilers noticed
> this, not the new fancy ones.
+1
I'm glad the buildfarm is so diverse.
--
Vik Fearing