Using Ordering to sort non-english strings in memory

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Using Ordering to sort non-english strings in memory

Hugi Thordarson
Hi all.

It looks like Cayenne's Ordering uses String.compareTo() when sorting String values in memory using orderList() and orderedList() — and there doesn’t seem to be any way to change that behaviour. This means sorting of international strings in memory is somewhat broken.

Do you think it would be a good idea to add support for:

1) Specifying a default java.text.Collator to use for new Ordering instances.
2) Specifying a java.text.Collator for an instance of Ordering.

What do you think?

Cheers,
- hugi
Reply | Threaded
Open this post in threaded view
|

Re: Using Ordering to sort non-english strings in memory

Andrus Adamchik
I am +1 in principle. Just need to figure out a clean implementation that can distinguish between String and other types of properties. Perhaps the initial solution may be a "StringOrdering extends Ordering" that takes the collator and overrides comparison method?

Andrus


> On Jan 11, 2017, at 5:12 PM, Hugi Thordarson <[hidden email]> wrote:
>
> Hi all.
>
> It looks like Cayenne's Ordering uses String.compareTo() when sorting String values in memory using orderList() and orderedList() — and there doesn’t seem to be any way to change that behaviour. This means sorting of international strings in memory is somewhat broken.
>
> Do you think it would be a good idea to add support for:
>
> 1) Specifying a default java.text.Collator to use for new Ordering instances.
> 2) Specifying a java.text.Collator for an instance of Ordering.
>
> What do you think?
>
> Cheers,
> - hugi

Reply | Threaded
Open this post in threaded view
|

Re: Using Ordering to sort non-english strings in memory

Hugi Thordarson
That would probably be ideal, since certain things like case sensitivity and collations apply only to Strings. But since case sensitivity is already a part of Ordering I propose the following solution. Quite simple, just checks if the values are strings and if so, uses a collator specified either per Ordering instance or globally:

https://github.com/godurkodi/cayenne/commit/61738f56dad753da07040b0f44de7bf3f297c680

Any comment?

Cheers,
- hugi


> On 11. jan. 2017, at 22:42, Andrus Adamchik <[hidden email]> wrote:
>
> I am +1 in principle. Just need to figure out a clean implementation that can distinguish between String and other types of properties. Perhaps the initial solution may be a "StringOrdering extends Ordering" that takes the collator and overrides comparison method?
>
> Andrus
>
>
>> On Jan 11, 2017, at 5:12 PM, Hugi Thordarson <[hidden email]> wrote:
>>
>> Hi all.
>>
>> It looks like Cayenne's Ordering uses String.compareTo() when sorting String values in memory using orderList() and orderedList() — and there doesn’t seem to be any way to change that behaviour. This means sorting of international strings in memory is somewhat broken.
>>
>> Do you think it would be a good idea to add support for:
>>
>> 1) Specifying a default java.text.Collator to use for new Ordering instances.
>> 2) Specifying a java.text.Collator for an instance of Ordering.
>>
>> What do you think?
>>
>> Cheers,
>> - hugi
>

Reply | Threaded
Open this post in threaded view
|

Re: Using Ordering to sort non-english strings in memory

Michael Gentry
Hi Hugi,

Instead of a global/static variable, perhaps we could make it part of a
custom ServerModule configuration?  That way you can create your
ServerModule with the collation defaults you require, which would make it
easier for a single application to have multiple ServerModules to support
different languages.

Thanks,

mrg


On Fri, Jan 20, 2017 at 8:03 AM, Hugi Thordarson <[hidden email]> wrote:

> That would probably be ideal, since certain things like case sensitivity
> and collations apply only to Strings. But since case sensitivity is already
> a part of Ordering I propose the following solution. Quite simple, just
> checks if the values are strings and if so, uses a collator specified
> either per Ordering instance or globally:
>
> https://github.com/godurkodi/cayenne/commit/61738f56dad753da07040b0f44de7b
> f3f297c680
>
> Any comment?
>
> Cheers,
> - hugi
>
>
> > On 11. jan. 2017, at 22:42, Andrus Adamchik <[hidden email]>
> wrote:
> >
> > I am +1 in principle. Just need to figure out a clean implementation
> that can distinguish between String and other types of properties. Perhaps
> the initial solution may be a "StringOrdering extends Ordering" that takes
> the collator and overrides comparison method?
> >
> > Andrus
> >
> >
> >> On Jan 11, 2017, at 5:12 PM, Hugi Thordarson <[hidden email]> wrote:
> >>
> >> Hi all.
> >>
> >> It looks like Cayenne's Ordering uses String.compareTo() when sorting
> String values in memory using orderList() and orderedList() — and there
> doesn’t seem to be any way to change that behaviour. This means sorting of
> international strings in memory is somewhat broken.
> >>
> >> Do you think it would be a good idea to add support for:
> >>
> >> 1) Specifying a default java.text.Collator to use for new Ordering
> instances.
> >> 2) Specifying a java.text.Collator for an instance of Ordering.
> >>
> >> What do you think?
> >>
> >> Cheers,
> >> - hugi
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Using Ordering to sort non-english strings in memory

Hugi Thordarson
Hi Michael,
that sounds sensible. Haven’t done that before though and not sure how to implement it. Could you point me towards an example implementation?

Cheers,
- hugi


> On 20. jan. 2017, at 16:38, Michael Gentry <[hidden email]> wrote:
>
> Hi Hugi,
>
> Instead of a global/static variable, perhaps we could make it part of a
> custom ServerModule configuration?  That way you can create your
> ServerModule with the collation defaults you require, which would make it
> easier for a single application to have multiple ServerModules to support
> different languages.
>
> Thanks,
>
> mrg
>
>
> On Fri, Jan 20, 2017 at 8:03 AM, Hugi Thordarson <[hidden email]> wrote:
>
>> That would probably be ideal, since certain things like case sensitivity
>> and collations apply only to Strings. But since case sensitivity is already
>> a part of Ordering I propose the following solution. Quite simple, just
>> checks if the values are strings and if so, uses a collator specified
>> either per Ordering instance or globally:
>>
>> https://github.com/godurkodi/cayenne/commit/61738f56dad753da07040b0f44de7b
>> f3f297c680
>>
>> Any comment?
>>
>> Cheers,
>> - hugi
>>
>>
>>> On 11. jan. 2017, at 22:42, Andrus Adamchik <[hidden email]>
>> wrote:
>>>
>>> I am +1 in principle. Just need to figure out a clean implementation
>> that can distinguish between String and other types of properties. Perhaps
>> the initial solution may be a "StringOrdering extends Ordering" that takes
>> the collator and overrides comparison method?
>>>
>>> Andrus
>>>
>>>
>>>> On Jan 11, 2017, at 5:12 PM, Hugi Thordarson <[hidden email]> wrote:
>>>>
>>>> Hi all.
>>>>
>>>> It looks like Cayenne's Ordering uses String.compareTo() when sorting
>> String values in memory using orderList() and orderedList() — and there
>> doesn’t seem to be any way to change that behaviour. This means sorting of
>> international strings in memory is somewhat broken.
>>>>
>>>> Do you think it would be a good idea to add support for:
>>>>
>>>> 1) Specifying a default java.text.Collator to use for new Ordering
>> instances.
>>>> 2) Specifying a java.text.Collator for an instance of Ordering.
>>>>
>>>> What do you think?
>>>>
>>>> Cheers,
>>>> - hugi
>>>
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Using Ordering to sort non-english strings in memory

Michael Gentry
Hi Hugi,

Sorry for the delay ...

To be honest, I'm not terribly familiar with the DI system (I'm still
mainly on 3.0.2), so I'd have to dig into the code to try to figure it
out.  Perhaps someone else can give you a quicker pointer or two than I can.

Thanks,

mrg


On Fri, Jan 20, 2017 at 1:47 PM, Hugi Thordarson <[hidden email]> wrote:

> Hi Michael,
> that sounds sensible. Haven’t done that before though and not sure how to
> implement it. Could you point me towards an example implementation?
>
> Cheers,
> - hugi
>
>
> > On 20. jan. 2017, at 16:38, Michael Gentry <[hidden email]> wrote:
> >
> > Hi Hugi,
> >
> > Instead of a global/static variable, perhaps we could make it part of a
> > custom ServerModule configuration?  That way you can create your
> > ServerModule with the collation defaults you require, which would make it
> > easier for a single application to have multiple ServerModules to support
> > different languages.
> >
> > Thanks,
> >
> > mrg
> >
> >
> > On Fri, Jan 20, 2017 at 8:03 AM, Hugi Thordarson <[hidden email]>
> wrote:
> >
> >> That would probably be ideal, since certain things like case sensitivity
> >> and collations apply only to Strings. But since case sensitivity is
> already
> >> a part of Ordering I propose the following solution. Quite simple, just
> >> checks if the values are strings and if so, uses a collator specified
> >> either per Ordering instance or globally:
> >>
> >> https://github.com/godurkodi/cayenne/commit/
> 61738f56dad753da07040b0f44de7b
> >> f3f297c680
> >>
> >> Any comment?
> >>
> >> Cheers,
> >> - hugi
> >>
> >>
> >>> On 11. jan. 2017, at 22:42, Andrus Adamchik <[hidden email]>
> >> wrote:
> >>>
> >>> I am +1 in principle. Just need to figure out a clean implementation
> >> that can distinguish between String and other types of properties.
> Perhaps
> >> the initial solution may be a "StringOrdering extends Ordering" that
> takes
> >> the collator and overrides comparison method?
> >>>
> >>> Andrus
> >>>
> >>>
> >>>> On Jan 11, 2017, at 5:12 PM, Hugi Thordarson <[hidden email]>
> wrote:
> >>>>
> >>>> Hi all.
> >>>>
> >>>> It looks like Cayenne's Ordering uses String.compareTo() when sorting
> >> String values in memory using orderList() and orderedList() — and there
> >> doesn’t seem to be any way to change that behaviour. This means sorting
> of
> >> international strings in memory is somewhat broken.
> >>>>
> >>>> Do you think it would be a good idea to add support for:
> >>>>
> >>>> 1) Specifying a default java.text.Collator to use for new Ordering
> >> instances.
> >>>> 2) Specifying a java.text.Collator for an instance of Ordering.
> >>>>
> >>>> What do you think?
> >>>>
> >>>> Cheers,
> >>>> - hugi
> >>>
> >>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Using Ordering to sort non-english strings in memory

Hugi Thordarson
Hi all,
anyone have a suggestion for how this should be implemented, i.e. where to set Ordering's Locale globally? I’d love to get these modifications into master.

Cheers,
- hugi



> On 28. jan. 2017, at 12:50, Michael Gentry <[hidden email]> wrote:
>
> Hi Hugi,
>
> Sorry for the delay ...
>
> To be honest, I'm not terribly familiar with the DI system (I'm still
> mainly on 3.0.2), so I'd have to dig into the code to try to figure it
> out.  Perhaps someone else can give you a quicker pointer or two than I can.
>
> Thanks,
>
> mrg
>
>
> On Fri, Jan 20, 2017 at 1:47 PM, Hugi Thordarson <[hidden email]> wrote:
>
>> Hi Michael,
>> that sounds sensible. Haven’t done that before though and not sure how to
>> implement it. Could you point me towards an example implementation?
>>
>> Cheers,
>> - hugi
>>
>>
>>> On 20. jan. 2017, at 16:38, Michael Gentry <[hidden email]> wrote:
>>>
>>> Hi Hugi,
>>>
>>> Instead of a global/static variable, perhaps we could make it part of a
>>> custom ServerModule configuration?  That way you can create your
>>> ServerModule with the collation defaults you require, which would make it
>>> easier for a single application to have multiple ServerModules to support
>>> different languages.
>>>
>>> Thanks,
>>>
>>> mrg
>>>
>>>
>>> On Fri, Jan 20, 2017 at 8:03 AM, Hugi Thordarson <[hidden email]>
>> wrote:
>>>
>>>> That would probably be ideal, since certain things like case sensitivity
>>>> and collations apply only to Strings. But since case sensitivity is
>> already
>>>> a part of Ordering I propose the following solution. Quite simple, just
>>>> checks if the values are strings and if so, uses a collator specified
>>>> either per Ordering instance or globally:
>>>>
>>>> https://github.com/godurkodi/cayenne/commit/
>> 61738f56dad753da07040b0f44de7b
>>>> f3f297c680
>>>>
>>>> Any comment?
>>>>
>>>> Cheers,
>>>> - hugi
>>>>
>>>>
>>>>> On 11. jan. 2017, at 22:42, Andrus Adamchik <[hidden email]>
>>>> wrote:
>>>>>
>>>>> I am +1 in principle. Just need to figure out a clean implementation
>>>> that can distinguish between String and other types of properties.
>> Perhaps
>>>> the initial solution may be a "StringOrdering extends Ordering" that
>> takes
>>>> the collator and overrides comparison method?
>>>>>
>>>>> Andrus
>>>>>
>>>>>
>>>>>> On Jan 11, 2017, at 5:12 PM, Hugi Thordarson <[hidden email]>
>> wrote:
>>>>>>
>>>>>> Hi all.
>>>>>>
>>>>>> It looks like Cayenne's Ordering uses String.compareTo() when sorting
>>>> String values in memory using orderList() and orderedList() — and there
>>>> doesn’t seem to be any way to change that behaviour. This means sorting
>> of
>>>> international strings in memory is somewhat broken.
>>>>>>
>>>>>> Do you think it would be a good idea to add support for:
>>>>>>
>>>>>> 1) Specifying a default java.text.Collator to use for new Ordering
>>>> instances.
>>>>>> 2) Specifying a java.text.Collator for an instance of Ordering.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>> Cheers,
>>>>>> - hugi
>>>>>
>>>>
>>>>
>>
>>