863966 - Consider creating some sort of parsed-selector cache for querySelector(All)

Status: RESOLVED FIXED

(Core :: DOM: Core & HTML, defect)

Description

[Boris Zbarsky [:bzbarsky]]

Need to figure out what, if anything, needs to go into the cache key other than the obvious selector string.  Performance testcase:

<script>
  var el = document.createElement("form");
  var count = 10000000;
  var start = new Date;
  for (var i = 0; i < count; ++i)
    el.querySelector("a");
  var end = new Date;
  document.write((end - start) / count * 1e6);
</script>

Comment 1

[Boris Zbarsky [:bzbarsky]]

Mina, if you get bored.... ;)

Comment 2

[Mina Almasry]

Always :-D

Comment 3

[Mina Almasry]

Attachment: make-errors

I'm working on this, but I've hit something of a wall. I believe what you want is to add a hashtable for each node, that hashes the keys to an nsSimpleContentList. Something like:

ns(RefPtr|Class)Hashtable <nsStringHashKey, nsSimpleContentList>

My problem seems to be that nsSimpleContentList can't work with any hashtable. Any time I define a hashtable like that I get strange make errors like in the attachment. What to do?

Comment 4

[:Ms2ger (he/him; ⌚ UTC+1/+2)]

You can't include nsIContent.h in nsINode.h; that'd cause an include cycle. It looks like you probably can include nsContentList.h, but that's probably not a good idea given all the work we're doing to reduce includes. See if you can get away with just including nsRefPtrHashtable.h and forward-declaring everything else?

+  struct CacheEntry
+  {
+    nsSimpleContentList mList;
+    bool mAreAll;
+  };

This won't work; mList should be a pointer. (Though you don't seem to actually use it?)

Comment 5

[Mina Almasry]

Attachment: Patch

Comment 6

[Mina Almasry]

Attachment: test-crash

Comment 7

[Mina Almasry]

Attachment: WIP patch

Comment 8

[Mina Almasry]

So I got passed that and here is a WIP patch. The patch builds, but the tests fail with a memory crash, which makes me think I have a memory leak somewhere or this method of caching takes too much memory. I'm still looking for the cause, but I thought I'd post the patch here anyway to check that this is what you want, and if it's obvious where the leak is please let me know. I think I'm using the nsClassHashtable correctly..

Comment 9

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 791872 [details] [diff] [review]
WIP patch

So first off, adding a hashtable to every single node isn't desirable: it uses too much memory.

Why can't we use a static, or per-document, hashtable here?  Furthermore, I suspect we really want something using nsExpirationTracker so we don't leak the memory for the lifetime of the document, or worse yet the process.

Your memory errors come from explicitly deleting the refcounted list in ~CacheEntry.  You'd probably just want to make mList an nsRefPtr instead.

But more to the point, we don't want to cache the _list_.  We want to produce a new list each time, not least because its contents will change as the DOM is modified.  What we want to cache is the nsCSSSelectorList*.  The new setup would probably have the cache own it and just return an nsCSSSelectorList* where we currently have an nsAutoPtr<nsCSSSelectorList>...

Comment 10

[Mina Almasry]

Attachment: Patch v1

My bad I was way off. Better?

Comment 11

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 795091 [details] [diff] [review]
Patch v1

Better, but clearly untested: for example, nothing in this patch ever sets mSelectorCache, so trying to use querySelector with this patch crashes.

I think it's probably simpler to just make mSelectorCache be a SelectorCache object, not a pointer to a SelectorCache object.

>+      nsClassHashtable<nsStringHashKey, nsCSSSelectorList> mTable;

I don't like making this public.  Better for it to be private and 
have a method to look up a given string on a SelectorCache.

It's worth explicitly documenting that we're not calling MarkUsed because it would just slow down lookups and because we're OK expiring things after a few second even if they're being used.

>+  nsAutoPtr<SelectorCache> mSelectorCache;

Likewise, this should not be public.  Instead, have a method for getting a SelectorCache& or something?

CacheSelectorList should be documented as taking ownership of aSelectorList.

>+    if (NS_FAILED(rv)) {

Then I think you need to delete selectorList.

I'd like to see a patch with the above fixed, and some numbers on this bug's performance testcase...

Comment 12

[Boris Zbarsky [:bzbarsky]]

Mina, are you going to have time to wrap this up?

Comment 13

[Mina Almasry]

Yes. Sorry for the delay but I'm back on it.

If I'm holding anyone back with the delay feel free to assign it to someone else. I'll understand :)

Comment 14

[Boris Zbarsky [:bzbarsky]]

No one is blocked on this right now.

Comment 15

[Boris Zbarsky [:bzbarsky]]

Well, I was slightly wrong.  Some benchmarks are blocked on it.  jandem, what's our timeframe for peacekeeper?

Comment 16

[Jan de Mooij [:jandem]]

(In reply to Boris Zbarsky [:bz] from comment #15)
> Well, I was slightly wrong.  Some benchmarks are blocked on it.  jandem,
> what's our timeframe for peacekeeper?

The browser grand prix and other websites use peacekeeper to measure "JS speed" and winning the grand prix is one of our goals. That's why I'm looking into peacekeeper and browsermark this week. So the timeframe is "ASAP"...

If we could finish and land this soonish I'd appreciate it :)

Comment 17

[Mina Almasry]

Attachment: Patch v2

Sorry for the delay. Here is an updated patch.

Comment 18

[Mina Almasry]

Attachment: Patch v3

Added the MarkUsed doc string.

Comment 19

[Mina Almasry]

Attachment: no-caching.html

Performance of querySelector without caching. Time in microseconds. Note that count is 1,000,000 and not 10,000,000. I hope that's okay.

Comment 20

[Mina Almasry]

Attachment: querySelector-4gen-1000.html

Performance with caching. 14,320 ns was improved to 3,175 ns.

Comment 21

[Mina Almasry]

Attachment: Interdiff

This is the interdiff, too.

My bad the time in the tests is in nanoseconds, not milliseconds.

Comment 22

[Mina Almasry]

Attachment: no-caching.html

Comment 23

[Mina Almasry]

Attachment: querySelector-4gen-1000.html

Comment 24

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 806569 [details] [diff] [review]
Patch v3

>+++ b/content/base/public/nsIDocument.h
>+private:
>+  class SelectorCacheKey

So I think the private class declarations should live over with the other private functions on nsIDocument, the member should live next to other members, and the public function (and a forward-declaration of class SelectorCache) should be next to other public functions.

>+      SelectorCacheKey(const nsAString& aString) : mKey(aString) { }

This should have MOZ_COUNT_CTOR/MOZ_COUNT_DTOR bits.

>+      void CacheList(const nsAString& aSelector, nsCSSSelectorList* aSelectorList);

This needs documentation about the fact that it takes ownership of aSelectorList.

>+      // We do not call MarkUsed because it would just slow down lookups and

This comment should be up in GetList.

>+++ b/content/base/src/nsINode.cpp
>   NS_ENSURE_TRUE(selectorList, NS_OK);

This needs to be before the CacheList call.

r=me with those issues fixed.  Thank you!

Comment 25

[Mina Almasry]

Attachment: Patch v4

Try: https://tbpl.mozilla.org/?tree=Try&rev=3adda0209d44

I need another review I think. I had to move SelectorCache's constructor from nsIDocument.h to nsDocument.cpp. The reason is nsCSSSelectorList is forward declared, and therefore the template instantiation should be in the .cpp file.

Comment 26

[Boris Zbarsky [:bzbarsky]]

> The reason is nsCSSSelectorList is forward declared, and therefore the template
> instantiation should be in the .cpp file.

I don't follow this.  Is the issue that the mTable member's constructor wants to see the declaration of nsCSSSelectorList or something?  What failed with the old approach?

Comment 27

[Mina Almasry]

Attachment: make-errors

Here is the make error I get.

The instantiation of mTable contains a call to delete nsCSSSelectorList, and so nsCSSSelectorList needs to be a complete type by then. The instantiation happens at the constructor of SelectorCache, so the constructor can't be in nsIDocument.h (nsCSSSelectorList is only forward declared there) but can be in nsDocument.cpp.

This is my fault big time. It looks like the patch that was granted a review was the wrong patch, and it didn't build. But the difference between that patch and the correct one is only moving the constructor.

Comment 28

[Boris Zbarsky [:bzbarsky]]

Ah, I see.  It's the instantiation of the clearEntry hook that's relevant.

Comment 29

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 808898 [details] [diff] [review]
Patch v4

r=me

Comment 30

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/d20b53c907ee

Comment 31

[Ryan VanderMeulen [:RyanVM]]

Backed out for leaks.
https://hg.mozilla.org/integration/mozilla-inbound/rev/004b170734ef

https://tbpl.mozilla.org/php/getParsedLog.php?id=28284916&tree=Mozilla-Inbound

Comment 32

[Boris Zbarsky [:bzbarsky]]

I wonder whether ~nsExpirationTracker expires everything that's left in it.  If it doesn't, we could be leaking SelectorCacheKey.  We'd know for sure if my review comments on that (MOZ_CONT_CTOR/DTOR) had gotten addressed and if I hadn't missed pointing the problem out again the second time around.  :(

One other thing: reading the expiration tracker code more closely, it looks like it could expire a cached entry while we're selector-matching.  We should probably make our NotifyExpired do the delete async...

Comment 33

[Boris Zbarsky [:bzbarsky]]

What we should consider doing is AgeAllGenerations() in ~SelectorCache and seeing if that helps.  

And a _debug_ try run.  Way more useful than an opt one, typically.

Comment 34

[Boris Zbarsky [:bzbarsky]]

Mina, ping?

Comment 35

[Mina Almasry]

Attachment: Patch v5

Sorry for the delay, again. Here is a test passing patch with the selector cache key deleted async.

Try push: https://tbpl.mozilla.org/?tree=Try&rev=910f3f415d75

Comment 36

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 812486 [details] [diff] [review]
Patch v5

>+#include "nsThreadUtils.h"

Let's not do that.

Instead, please move the (virtual, so it's not like it's getting inline anyway, right?) NotifyExpired to the .cpp file.  And mark it as MOZ_OVERRIDE while we're here.

>+        nsIDocument::SelectorCacheKeyDeleter asyncDelete(aSelector);

No, this is wrong.  You want to actually post the event.  Something like this:

  NS_DispatchToCurrentThread(new SelectorCacheKeyDeleter(aSelector));

r=me with those changes.

Comment 37

[Mina Almasry]

Attachment: query-selector-from-bugzilla.patch

I made these changes, but now the tests fail. The fails are mainly leaks of SelectorCacheKey.

https://tbpl.mozilla.org/?tree=Try&rev=238684cf2358

What to do here? I think with the mistake I made in the patch in comment #35, the SelectorCacheKey gets deleted synchronously and that passes the tests. Can we maybe leave it that way and call MarkUsed() instead to keep nsExpirationTracker from expiring while we are doing matching?

Comment 38

[Boris Zbarsky [:bzbarsky]]

You can't use MarkUsed to prevent expiration due to memory-pressure notifications, which were the thing that could cause dangling pointers.

What's confusing me is that we don't show any leaks of SelectorCacheKeyDeleter, so presumably the event in fact got posted.  I wonder whether we just manage to take our leak measurement while the event is still pending or something...

Can you reproduce the leak locally?

Comment 39

[Mina Almasry]

Attachment: patch v6

Okay SelectorCacheKeyDeleter was leaking, I just didn't have the CTOR/DTOR bits there.

The problem was the I have to save the nsRunnable instance into an nsCOMPtr before passing it into NS_DispatchToCurrentThread. It now passes the tests again: https://tbpl.mozilla.org/?tree=Try&rev=e3e6e4c2cd34

While you're at it could you please explain why does this work? Why does doing the delete in NotifyExpired async make it that AgeAllGenerations() called with a memory-pressure won't expire things while we are selector matching?

Why did I need to save new SelectorCacheKeyDeleter into an nsCOMPtr? I thought the new SelectorCacheKeyDeleter would start with refcount 0, then get incremented to 1 when it is passed to NS_DispatchToCurrentThread, then get decremented back to 0 and released after it runs which is what we want.

Thanks!

Comment 40

[Boris Zbarsky [:bzbarsky]]

> Okay SelectorCacheKeyDeleter was leaking,

Huh.  I wonder why that leak wasn't getting reported...  Oh, because there was no initial addref!

So what happened there is that the thread had already shut down, so  NS_DispatchToCurrentThread didn't do anything useful with the argument.  Fun.

> Why does doing the delete in NotifyExpired async make it that AgeAllGenerations()
> called with a memory-pressure won't expire things while we are selector matching?

So the worry I had is that we would look up a selector in the cache, start selector matching, allocate some memory during selector matching, hit a low-memory condition, get a memory-pressure notification, the expiration tracker would expire all the stuff in it, delete the selector.  Then we unwind the stack back to selector matching, try to work with the now-deleted selector object and crash.

If we do the delete async, that ensures that we unwind out of selector matching before the selector gets deleted.

> then get incremented to 1 when it is passed to NS_DispatchToCurrentThread,

Nope.  nsThread::Dispatch checks for thread shutdown before taking a reference to the argument.

So with the stack ref, the delete will happen async unless we're in shutdown, in which case it'll happen sync when the nsCOMPtr goes off the stack.  Which is fine, because if we're at that point in shutdown we're not in the middle of a querySelector call, I would hope.  ;)

Comment 41

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 813202 [details] [diff] [review]
patch v6

NotifyExpired should still be marked virtual and MOZ_OVERRIDE.

r=me with that.

Comment 42

[Mina Almasry]

Attachment: patch v7

Extra thanks for your patience on this one Boris.

Comment 43

[Mina Almasry]

Try push in comment #39. Changes in patch from that version to this one are trivial and I made sure it compiles locally. If you need another push, just let me know. Thanks!

Comment 44

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/99b749ea1fb9

Comment 45

[Phil Ringnalda (:philor)]

Backed out in http://hg.mozilla.org/integration/mozilla-inbound/rev/97bdee4bd259. I still have retriggers running on the pushes before to be sure, but I expect this'll wind up being the cause of the frequent Android robocop-2 crashes like https://tbpl.mozilla.org/php/getParsedLog.php?id=28757488&tree=Mozilla-Inbound. Unfortunately, your -b d Try run didn't run any Android (or b2g) tests, since we only run them on opt builds. If anyone ever hassles you about the "waste" of running -b do -p all -u all try runs, refer them to me, I'll yell until they wish they'd never brought it up.

Comment 46

[Boris Zbarsky [:bzbarsky]]

Hmm.  So that's a memory-pressure notification coming through, which calls sExpirationTracker<nsIDocument::SelectorCacheKey, 4u>::ExpirationTrackerObserver::Observe and that lands us in AgeAllGenerations.

The actual crash is in the nsTarray_base::Length() call that AgeOneGeneration does...

So how did we manage to end up with a bogus tarray there?  Might be worth adding logging to ~nsExpirationTracker and ExpirationTrackerObserver::Destroy and ExpirationTrackerObserver::Observe to log the mOwner and this and mObserver object pointers, just to see whether we're getting notified on a destroyed object somehow.

Comment 47

[Mina Almasry]

I will look at this more soon and to see if I have a proposal on what to do, but so far I have:

this try push: https://tbpl.mozilla.org/?tree=Try&rev=8d60741bf8b6

With this log (search for MINA): https://tbpl.mozilla.org/php/getParsedLog.php?id=28812924&tree=Try&full=1

Just before the crash we call AgeAllGenerations on 0x59569838 from the observer 0x5be2acb0, but a little above that we enter the destructor of 0x59569838, and we call Destroy on 0x5be2acb0.

My guess is that queued notifications still get processed after Destroy removes the event listener, or that the event listener is removed while a memory pressure notification is being processed so there is a race there.

Comment 48

[Josh Matthews [:jdm]]

It sure looks to me like the observer service grabs all the observers that exist when NotifyObservers is called (http://mxr.mozilla.org/mozilla-central/source/xpcom/ds/nsObserverService.cpp#319 and http://mxr.mozilla.org/mozilla-central/source/xpcom/ds/nsObserverList.cpp#88), so if the expiration tracker were destroyed during a memory-pressure notification, the observer service would not notice the removal of the tracker's memory-pressure observer. I'm surprised this hasn't ever come up before.

Comment 49

[Boris Zbarsky [:bzbarsky]]

Yeah, we should have Destroy null out mOwner and have Observe null-check it.

Comment 50

[Mina Almasry]

Attachment: patch v8

Great, that worked. Try: https://tbpl.mozilla.org/?tree=Try&rev=aace95d87de0

Comment 51

[Boris Zbarsky [:bzbarsky]]

Comment on attachment 814675 [details] [diff] [review]
patch v8

r=me

Comment 52

[Mina Almasry]

Okay that try was -b do -p all -u all so I think I'm good here. Marking checkin needed. Fingers crossed :(

Comment 53

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/965b4f247f4b

Comment 54

[Boris Zbarsky [:bzbarsky]]

Yay:

  (Improvement) Mozilla-Inbound-Non-PGO - Dromaeo (CSS) - Ubuntu HW 12.04 x64 - 3.85%

Comment 55

[Mina Almasry]

Yay!

Comment 56

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/965b4f247f4b